{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实验1：典型监督学习方法分类实践与比较分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 实验目的与要求\n",
    "1. 利用所学习的监督学习方法完成目标识别实验方案的设计。\n",
    "2. 编程并利用相关软件完成实验测试，得到实验结果。\n",
    "3. 通过对实验数据的分析, 整理, 方法的对比, 得出实验结论, 培养学生创新思维和编写实验报告的能力, 以及处理一般工程设计技术问题的初步能力及实事求是的科学态度。\n",
    "4. 利用实验更加直现、方便和易于操作的优势，提高学生学习兴趣，让学生自主发挥设计和实施实验，发挥学生潜在的积极性和创造性。\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 实验主要内容\n",
    "1. 采用已经学过的监督学习的方法，如：逻辑回归、决策树、神经网络等实现分类任务。\n",
    "2. 分析比较不同方法的优缺点\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据源\n",
    "\n",
    "| 数据集特征 | 多变量 | 实例数  | 150  |    分区    |    life    |\n",
    "| :--------: | :----: | :-----: | :--: | :--------: | :--------: |\n",
    "|  属性特征  |  Real  | 属性数  |  4   |  捐赠日期  | 1988-07-01 |\n",
    "|  相关任务  |  分类  | 缺失值? |  无  | 网页点击数 |  4121103   |\n",
    "\n",
    "[数据源](http://archive.ics.uci.edu/ml/datasets/Iris)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据集信息\n",
    "\n",
    "This is perhaps the best known database to be found in the pattern recognition literature. Fisher's paper is a classic in the field and is referenced frequently to this day. (See Duda & Hart, for example.) The data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant. One class is linearly separable from the other 2; the latter are NOT linearly separable from each other.\n",
    "\n",
    "这可能是模式识别文献中最著名的数据库。Fisher 的论文是该领域的经典之作，至今仍被频繁引用(例如，参见 Duda & Hart。）该数据集包含 3 个类别，每个类别 50 个实例，其中每个类别指的是一种鸢尾属植物。一类与另两类是线性可分的；后者不是线性可分离的。\n",
    "\n",
    "Predicted attribute: class of iris plant.    \n",
    "预测属性：鸢尾属植物类。\n",
    "\n",
    "This is an exceedingly simple domain.    \n",
    "这是一个非常简单的领域。\n",
    "\n",
    "This data differs from the data presented in Fishers article (identified by Steve Chadwick, spchadwick '@' espeedaz.net ). The 35th sample should be: 4.9,3.1,1.5,0.2,\"Iris-setosa\" where the error is in the fourth feature. The 38th sample: 4.9,3.6,1.4,0.1,\"Iris-setosa\" where the errors are in the second and third features.\n",
    "\n",
    "该数据与 Fishers 文章中的数据不同（由 Steve Chadwick、spchadwick '@' espedaz.net确定）。第35个样本应为：4.9,3.1,1.5,0.2，“Iris setosa”，其中误差在第四个特征中。第38个样本：4.9,3.6,1.4,0.1，“Iris setosa”，其中错误出现在第二和第三个特征中。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 属性信息\n",
    "\n",
    "1. sepal length in cm    \n",
    "   萼片长度\n",
    "2. sepal width in cm    \n",
    "   萼片宽度\n",
    "3. petal length in cm      \n",
    "   花瓣长度 \n",
    "4. petal width in cm    \n",
    "   花瓣宽度\n",
    "5. class:      \n",
    "   类别    \n",
    "-- Iris Setosa  山鸢尾    \n",
    "-- Iris Versicolour  变色鸢尾    \n",
    "-- Iris Virginica  弗吉尼亚鸢尾    \n",
    "\n",
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据可视化\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       sepal_length  sepal_width  petal_length  petal_width\n",
      "count    150.000000   150.000000    150.000000   150.000000\n",
      "mean       5.843333     3.054000      3.758667     1.198667\n",
      "std        0.828066     0.433594      1.764420     0.763161\n",
      "min        4.300000     2.000000      1.000000     0.100000\n",
      "25%        5.100000     2.800000      1.600000     0.300000\n",
      "50%        5.800000     3.000000      4.350000     1.300000\n",
      "75%        6.400000     3.300000      5.100000     1.800000\n",
      "max        7.900000     4.400000      6.900000     2.500000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEFCAYAAADDkQ0WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbZUlEQVR4nO3df7DcdX3v8ecLhJkQkCQmc0JaILX8UDACElOghB7QjHCVaq1X2lJoRJuppd4CcUq8MjSt0gICwy0C96ZCoWrrZZgBWwNYlJwhgggJP1UQuBpgAqGDg3BDr9WE9/3j+zlkszlnz+737O73+9l9PWYy2R/f/X7fu+ez7/M5n5+KCMzMLB+7VR2AmZl1xonbzCwzTtxmZplx4jYzy4wTt5lZZpy4zcwy48TdQ5JWS1rdwfHLJd3Qu4h2ud5CSZv6dT0bLp2W/xbn+bSkT7d4fkzSaIvnRyWNTTeOOnHiHiKSzpE0q+o4zDoREZdFxGXtHt+NXxZ158Q9XM4BZlUcg1mv/WXVAfTaUCduSX8l6XlJL0j60/TYxyQ9Kek5SX+cHlst6RuSHpf0rKRTm86xOT1+Rg9iXCJpQ7rGGhUWStok6TxJWyQ9Kml+On5U0o8lPSHpRkl3SPozSVuA/YEH0mv2arjGLuexwVeX8i/pTkm/KelKSddLOlzSAw3P79LkIunCFPftwJvTY5ekck4qzz9oes3lkl6SdLekGWVirYuhTdyS5gDnA28H3gG8R9I7gPOAdwNHAqsljaSXHAYcB5wKXC9pL0kHAEuBQ4BjgC90OcY9ga8CfwQcAPwa8KH09AJgfvr/WeD30+OXAJ8E/hz49Yg4OSK+GBHzgeeAd0fE/Ij4jynOYwOsZuX/wXSOWRQ56dD02GSxLwHOSjFdmGIlIs5P5ZxUxg9veNmxwE+A/SgS/cklY62FN1UdQIVeAZ4C/gdwB0Vy/BjwVuBH6ZgZFIUI4NaIeBl4WdKLwCER8bCkc4CVwInACN11KLAQ+Ha6vydFYX0ICGB1RLwuaSOwbzrm5+k4gN3buMZk57HBVqfy/yBwFEU+CuBttEjcFL9A1qZ4HpD0aBvXeBG4OiJC0iNkXs6HtsYdEdspahY3A79FkQwF/GP6bT0f+FXgvvQSNbx8N+B1SUuBW4AfA8t7EKaApxviWQBcmZ7b0lBrblwp7AfA31LUfibtiW8w2XlsgNWs/D+YYnkN2AK8h9aJW+xcVl9v4xo/iR0r6mVfzoc2cUs6hKIm+22KPxnnUxTeUyTNl7QP8AhFDRfgQ5JmSzoKeAtFbeU3gPspmjNO6UGYTwB7SVoqaTfgy8An0nO7FD5JuwO/DSyJiLdFxD1Nh7wEHJjayedMdh4bfDUr/08DR6dzPknR/NKqFn1/inNfSe8Cjmh6/qeSDpS0h6TxmvVAlfOhTdwR8SSwnqLd60ngixGxHvgc8F3ghxR/Wj2cXvIQ8B3g68BZEfH/KGor7wA2UxTwrekL0a0YfwGcBlyVrvFz4NoWx28HHgaeTZ1Fd0la1HDIhcANFAn8fd2K0/JTp/KfasKPpDieBB6PiP9scfw9wNfSsVcDjzcd8hfAPcALwDs7jScH8nrcUxvv0Y6I1dVG0pqkxcBFFB0vomgq2S8izq00MMtaLuV/mAxz52TfpSFO+0/w1IGtahgdeAr4BfA8sB3YxI6mFbNK9aH8Dw3XuM3MMjO0bdxmZrly4jYzy0zP27jnzp0bCxcu7PVlWnrttdeYOXNmpTE0qls8UP+YNm7c+FJEzKs4pLa0KvN1/Jy7xe+tu1qW+Yjo6b+jjz46qrZu3bqqQ9hJ3eKJqH9MwIbocVnt1r9WZb6On3O3+L11V6sy76YSM7PMOHGbmWXGidvMLDOegNPCwlVrO37Npovf34NIzOpr/HuyctE2lrf5nfH3ZHpc4zYzy4wTt5lZZtxUUgNukjGzTrjGbWaWGSduM7PMOHGbmWXGidvMLDNO3GZmmXHiNjPLjBO3mVlmnLjNzDLjxG1mlpnsZk6WmWV4w8mDuSuHmQ0n17jNzDLjxG1mlpnsmkrM+kHSHOBo4KGIeKnqeAaNF1abHte4zZpI2g9YCywB1kmaJ+k6SfdKuqDi8MycuM0mcDhwbkRcBHwTOAnYPSKOAxZIOrjS6GzouanErElEfAtA0gkUte45wE3p6buA44GnGl8jaQWwAmBkZISxsbEJz71169ZJn8vVykXbABiZseN2L1T5udXt5+bEbTYBSQJOA34JCNicnnoVOKj5+IhYA6wBWLx4cYyOjk543rGxMSZ7LlfLG/acvPyx3qWUTaeP9uzcU6nbz23KphJJ+0q6XdKdkm6RtKfb+2zQReFs4F7gGGBGempv3MRoFWunAJ4OXBERy4AtwO/h9j4bYJLOl3RmujsLuJiieQTgCGBTBWGZvWHKv2si4pqGu/OAPwSuTPen1d5XRpk2tLLtU2Wu1c51muPp1XU6Ubc2PKg0pjXATZI+AXwfuBW4W9IC4BSKGrhZZdpukJJ0LDCborbRlfa+MpaXnPJeJoYy12qnHa65vaxX1+lE3drwoLqYIuJlYFnjY5JG02OXRsQrfQ/KrEFbbXVpMsJVwFnAVtzeZ0MmIl6OiJsiYkvVsZi10zm5J8VQqM9ExDPARtzeZ2ZWmXZqyx+nmPr7WUljFEOjzpB0BfBRihlmZmbWJ+10Tl4LXNv4mKR/we19ZmaVKDVaPnXe3DTlgWZm1nWeOZkpr65mNrw8IsTMLDNO3GZmmXFTiZm9oUwTnPWfa9xmZplx4jYzy4wTt5lZZtzG3WXttBGuXLSt1MJSZmbgxG02sNzROLjcVGJmlhknbrMm3q7P6s6J22xX3q7Pas1t3GZNymzXZ9ZPQ5G4H9v8ikdxWMc62a6v3X1W+7mPZpm9TKdjZEZvr1nlnqh125N1KBK3Wacatuv7XeA8ptiur919Vvu5j2a/KysrF23j8sd6l1K6vc9qJ+q2J6vbuM2aeLs+qzsnbrNdebs+qzU3lZg18XZ9VndO3GZt8HZ9ViduKjEzy4wTt5lZZpy4zcwy48RtZpYZJ24zs8y0lbgljUhan27vIekbaaW0s3obnpmZNZsycUuaDdwIzEwPfQrYkFZK+4CkfXoYn5mZNWmnxr0dOI1icR2AUXaMZ70XWNz9sMzMbDJTTsCJiFcBJI0/NJOdV0obaX5NuyullVFm9bFer1rWqariafVzqNvqZ1DPmMzqoMzMya0UK6W9QrFS2tbmA9pdKa2MMiue9XrVsk5VFU+r1dXqtvoZ1DMmszooM6rEK6WZmVWoTLXvRuA2SUuBw4Dvlb24d6E2M+tc2zXuiBhN/z9DsUraPcB7I2J7b0IzM7OJlGpojYjn8UppZmaV8MxJM7PM1GeohVmNSBoBbo6IpZL2AG4B5gBfiojrq43OOlGmL23Txe/vQSTd4xq3WRPPFra6c43bbFfjs4W/nu6PAqvS7fHZwusaX9DupLN+Tirq9ySvXk8sK/u5lYmp+Vp1mwzmxG3WpMxs4XYnnfVzUlGZyWrT0euJZa0mkLVS5nNovlbdJoO5qcRsauOzhaGYLezvjVXKNW6zqY3PFr6ZYrbwfdWGM5w8YW8HJ+4h0qrgr1y0bdI/Kcv0sA9YT37XZgubdYP/5DObhGcLW125xm3WBs8Wtjpx4rYpuW2xWv78rZmbSszMMuPEbWaWGSduM7PMOHGbmWXGidvMLDNO3GZmmfFwQLM+emzzK31f/MkGj2vcZmaZceI2M8uME7eZWWacuM3MMuPOSTOzJs3rw7Ra9ng6yi5l7Bq3mVlmnLjNzDJTOnFLuk7SvZIu6GZAZnXlMm91USpxS/owsHtEHAcskHRwd8MyqxeXeasTRUTnL5L+DrgjIm6T9BFgn4j4h4bnVwAr0t1DgR91I9hpmAu8VHEMjeoWD9Q/pgMjYl5VgXSxzNfxc+4Wv7fumrTMlx1VMhPYnG6/ChzU+GRErAHWlDx310naEBGLq45jXN3iAcfUhq6U+Zq9p67ye+ufsm3cW4EZ6fbe0ziPWS5c5q02yha+jcDx6fYRwKauRGNWXy7zVhtlm0puBdZLWgCcAhzTtYh6ozbNNknd4gHHNJVb6U6Zr9N76ja/tz4p1TkJIGk2sAy4OyK2dDUqsxpymbe6KJ24zcysGu5gMbNpkzRH0jJJc6uOZRgMbOKW9CZJz0oaS/8WVR3TOEnXSDq16jgAJH2y4TN6WNL/qkFMsyXdJmm9pP9ZdTzdIGlfSbdLulPSLZL2rDqmbpG0H7AWWAKsk1TZePtekTQi6aGq4xg3sIkbeCfwzxExmv49VnVAAJKWAvMj4l+rjgUgIq4d/4yA9dSjE+YM4CsRsRTYR1Jtxs9Ow+nAFRGxDNgCnFxxPN10OHBuRFwEfBN4V8Xx9MJl7BgOWrlBTtzHAL8j6TuSviqp8iVsJe0B/D2wSdIHq46nkaRfAUYiYmPVsQA/BQ6VNAvYH3i22nCmLyKuiYg70915wL9XGU83RcS3IuI+SSdQ1Lq/W3VM3STpJOA1il+4tTDIifsB4Lci4njgZ8B/qTYcAM4EfghcCiyR9KmK42l0NnBt1UEk3wEOBv4b8ATwcrXhdI+kY4HZEXFf1bF0kyQBpwG/BLZXHE7XpCatC4FVVcfSaJAT96MR8UK6/QRFIqjaUcCaNJTsK8CJFccDgKTdgBMjYl3VsSR/A/xJRPw1xc/uYxXH0xWS5gBXAWdVHUu3ReFs4F7gA1XH00WrgKsj4mdVB9JokBP3lyUdIWl34HeAR6oOCHgaeGu6vRh4psJYGi0Fvld1EA32Ahaln91vANmPWU01t5uAz0REXX7uXSHpfElnpruzKP7CHRTvBc6WNAYcKelLFccDDHbi/mvgy8DDwHcj4lv9vLik1ZJWNz18HXCipLuBP6Xo8Bg/frmkG/oY30JJm9Ld9wF39+vabfhbik7SV4A5wD9XG05XfBw4GvhsGsFzWtUBddEa4IxUrncH/g0m/Q50TNKnJX26xfNjkkZbPD+aEm/HIuKEhs77hyPiE2XO022Vd9j1SkR8n2JkybSlTrLlEXHlNGP6v8B/7UZMnZJ0DnDDRH/yRcR/73tALUTE/RQjFQZGRFxLffoQOtbqOxARL1PMKO2JiLhs6qN2kLQ6Ilb3II7Rbp+zrEGucXfTLOCcimOYrnMo3odZGbPI5zvwl1UH0GtDmbjTn3DfkPR4mqRzanr8Y5KelPScpD9Oj/0TxQiV/SVtkXRHw3n+StLmdI4zehDnEkkb0jXWqLBQ0iZJ56V4HpU0Px0/KunHkp6QdKOkOyT9maQtFMPqHkiv2avhGrucxwZfnb4DaVLSb0q6UtL1kg6X9EBTrKubXnOhpBck3Q68OT12SSrrpDh/0PSayyW9JOluSbUZk13GUCbu5DDgOOBU4HpJ7wTOA94NHAmsljQSEX+QHnsuIuZHxMkAkg6g6NQ7hGLM+Be6GVzqzPoq8EfAAcCvAR9KTy8A5qf/nwV+Pz1+CfBJ4M+BX4+IkyPiixExH3gOeHd6D/8xxXlsONTlO/BgOscsipx0aHpsQpKWUIzMOYxiqN6RABFxfirrpDgbm9uOBX4C7EeR6LOeADWwbdxtuDW1zb0s6UXggxQjPsa3nJpBUYBenOjFEfFsajdeSTGsb6TL8R0KLAS+ne7vSVFQH6IYZbE6Il6XtBHYNx3z83QcFJ1EU5nsPDYc6vIdeJBiqOybKMrk22iRuCl+2axNsT8g6dE2rvEixbC+kPQImZf1Ya5xq+H2bhQjGP4x/aaeD/wqMOkkCRVT128Bfgws71F8TzfEswC4Mj23paHW3DhU7gcUIzK+AEzaC99gsvPYcKjLd+BBihr9+OzE99A6cYudy+vrbVzjJ7FjKdTsy/owJ+4PqVjM6CjgLcA64BRJ8yXtQzHu+7B07E+Bt0jaK/2bQTG++H6K5oxTehDfE8BekpaqmCDzZWB8KNIuBU/FmOffBpZExNsi4p6mQ14CDkzt5HMmO48Nlbp8B56mGCr5FPAkRfNLq1r0/SnOfSW9i2JHokY/lXSgpD0kjdesB6qsD3PifohiavXXgbPSIlSfo1hn4YcUf1Y9DG8M47sE+D/saCe7GXgHxQayhwFbJR3SreAi4hcUU4ivStf4OS2Gk0XEdoox68+mjqK7tPOKiBcCN1Ak8Pd1K07LWi2+A6km/AhF0n4SeDwi/rPF8fcAX0vHXg083nTIXwD3AC/QpSHBdTOUGymM91D3YqxnVVSsoHcRRaeLKJpK9ouIcysNzGppEL8Dw2SYOyf7Kg1v2n+Cpw5sVbvowFPAL4DnKRb52cSOphWzyvXhOzA0hrLGbWaWs2Fu4zYzy5ITt5lZZnrexj137txYuHBhry+zk9dee42ZM2f29ZrT5Zhb27hx40sRkcVehuNlPsefabv83nqvVZnveeJeuHAhGzZs6PVldjI2Nsbo6Ghfrzldjrk1SdmsYT1e5nP8mbbL7633WpV5N5WYmWXGidvMLDNO3GZmmal0As7CVWs7fs2mi9/fg0jMdiVpBLgjIo6SdB3wduC2iPh82XO6zFs3uMZtNrnLgBmSPgzsHhHHAQskHVxxXDbkPOXdbAKSTmLHMqOjFDu0A9wFHE+xxEDj8SuAFQAjIyOMjY2xdetWxsbGdjrvykXbOo6l+Rx1MNF7GxQ5vDcnbrMmafehCyl2HLoVmEmxAh7Aq8BBza+JiDUUu52zePHiGB0dnXBY2fIyTSWnj055TL/VZchcL+Tw3txUYrarVRRLmv4s3d9KsRsMwN74e2MVc43bbFfvBU6SdDbFfoYHUOzZeR/Fov0/mvylZr3nxG3WJCJOGL8taYxiZ6H1khZQ7PRyTEWhmQH+k8+spYgYjYhXKToo7wNOjIhXqo3Khp1r3GZtSDuK3zTlgWZ94Bq3mVlmnLjNzDLjxG1mlhknbjOzzDhxm5llpq3ELWmOpGWS5vY6IDMza23KxC1pP2AtsARYJ2mepOsk3Svpgp5HaGZmO2mnxn04cG5EXAR8EzgJL3FpZlaZKSfgRMS3ACSdQFHrnkOJJS4n0qslLnNYlrGZY7Zh4g0lpqetmZOSBJwG/BIQJZa4nEivlrjMYVnGZo7ZzNrVVuKOiADOlvQ54CN4icvKucZiNrza6Zw8X9KZ6e4s4GKK5hEolrjc1JPIzMxsQu3UuNcAN0n6BPB9ih1B7vYSl2Zm1Winc/JlYFnjY5JG02OXeolLM7P+KrWsq5e4NDOrjjsWzcwy48RtZpYZJ24zs8x467IaWLhqLSsXbSs1IcnMho9r3GZmmXHiNjPLjBO3mVlmnLjNzDLjxG1mlhknbjOzzDhxm5llxonbzCwzTtxmZplx4jYzy4wTt5lZZqZcq0TSvsDX0rFbKTYNvhZ4O3BbRHy+pxGamU1Dp/uzrly0jdHehNI17dS4TweuiIhlwBbg94DdI+I4YIGkg3sZoJmZ7aydrcuuabg7D/hD4Mp0/y6KjYOfanyNpBXACoCRkRHGxsYmPPfKRds6jXfSczXaunVrW8fVxcpF2xiZUe7z6ES3P5PcPmezQdH2sq6SjgVmU+zqvjk9/CpwUPOxEbGGYpNhFi9eHKOjoxOes8wypptOn/hcjcbGxpjsmnW0PC3revljvV1lt53PrhO5fc5mg6KtzklJc4CrgLMo2rlnpKf2bvccZmbWHVMmXUl7UmwM/JmIeAbYSNE8AnAERQ3cbGBI2lfS7ZLulHSLpD0lXSfpXkkXVB2fWTu15Y8DRwOflTQGCDhD0hXARwFv22KDxh3yVmvtdE5eSzH87w2S/gVYBlwaEa/0KDbrsk6HRY3bdPH7uxxJvXWrQ36izttedcj323Q7pvv5OXR6rZEZ9fzMG5XqDYuIlymaT8wG1nQ75CfqvO1Vh3y/Tbdjup+fQ6fXWrloGx+teae7OxbNJuAOeaszF0CzJu6Qt7pz4jbblTvkrdZ6O+PDLEPukLe6c+I2a4M75K1O3FRiZpYZJ24zs8w4cZuZZcaJ28wsM07cZmaZyW5USTvrbaxctG2naa7DttaGmQ227BK3mVkdlVnErWyl0k0lZmaZceI2M8uME7eZWWbaauOWNALcHBFLJe0B3ALMAb4UEdf3MkCr3mRtd82dwI3cIWzWO+3sOTkbuBGYmR76FLAhbeP0AUn79DA+MzNr0k6NeztwGvD1dH8UWJVu3wssBtY1vmCibZwmUmb7onaMzNj53HXfhmjlom27xJyDVjHX/TM3y1k7e06+CiBp/KGZ7LyN08gEr9llG6eJlNm+qB0rF23j8sd2vLU6bv3UaPmqtbvEnINWMdf9MzfLWZnOSW/jZGZWoTJJ19s4mZlVqMzf5jcCt0laChwGfK+7IZmZWSttJ+6IGE3/PyNpGUWt+8KI2N6j2LJUZtqrmdVL3b/HpXrDIuJ5vI2TmVkl8hrGYDaEytb++jUJqu6100HkESFmZplxjdvM3tBu7bnVcgfWe65xm5llxonbzCwzbioxsyy4E3QHJ27riX5u42Q2bNxUYmaWGSduM7PMOHGbmWXGbdwtuDPEzOrINW4zs8w4cZuZZcaJ28wsM07cZmaZceI2M8tM6VElkq4D3g7cFhGf715IZvWUW5n3qKjBVarGLenDwO4RcRywQNLB3Q3LrF5c5q1OFBGdv0j6O+COiLhN0keAfSLiHxqeXwGsSHcPBX7UjWA7MBd4qc/XnC7H3NqBETGvT9faRckyn+PPtF1+b703aZkv21QyE9icbr8KHNT4ZESsAdaUPPe0SdoQEYurun4Zjrn2Oi7zg/z5+L1Vq2zn5FZgRrq99zTOY5YLl3mrjbKFbyNwfLp9BLCpK9GY1ZfLvNVG2aaSW4H1khYApwDHdC2i7qismWYaHHO93UrnZX6QPx+/twqV6pwEkDQbWAbcHRFbuhqVWQ25zFtdlE7cZmZWDXewmJllZqASt6R9Jd0u6U5Jt0jas+qY2iVpRNJDVcfRCUnXSDq16jjqRtJ1ku6VdEHVsfRCKqvrq46j23LKHwOVuIHTgSsiYhmwBTi54ng6cRk7hpvVnqSlwPyI+NeqY6mTQZ9hmdr5b6QY1z5osskfA5W4I+KaiLgz3Z0H/HuV8bRL0knAaxSFpfYk7QH8PbBJ0gerjqdmRoGb0u272DGEcFBsB06jmIQ0UHLKHwOVuMdJOhaYHRH3VR3LVNKfYxcCq6qOpQNnAj8ELgWWSPpUxfHUSfMMy5EKY+m6iHg1Il6pOo5eyiF/DFziljQHuAo4q+pY2rQKuDoiflZ1IB04CliThsR9BTix4njqxDMsM5ZL/hioQpVqrzcBn4mIZ6qOp03vBc6WNAYcKelLFcfTjqeBt6bbi4FcPut+8AzLTOWUPwZqHLekTwJ/AzySHro2Iv53hSF1RNJYRIxWHcdUJO0DXE/RDLAH8JGI2Nz6VcNB0puB9cC3STMsB7FpIZey2omc8sdAJW6zOvAMS+s1J24zs8wMVBu3mdkwcOI2M8uME7eZWWacuM3MMuPEbWaWmf8P7fF5rKMSGxYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "dataset = pandas.read_csv('iris.data')  # 读取csv数据\n",
    "print(dataset.describe())\n",
    "#直方图 histograms\n",
    "dataset.hist()\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 逻辑回归\n",
    "\n",
    "- [温州大学《机器学习》课程课件（三）逻辑回归](https://zhuanlan.zhihu.com/p/361621889)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "----\n",
    "\n",
    "### 分类问题\n",
    "\n",
    "#### 监督学习的最主要类型\n",
    "\n",
    "- 分类（Classification） \n",
    "\n",
    "  > 标签离散\n",
    "\n",
    "  - 身高1.85m，体重100kg的男人穿什么尺码的T恤？\n",
    "  - 根据肿瘤的体积、患者的年龄来判断良性或恶性？ \n",
    "  - 根据用户的年龄、职业、存款数量来判断信用卡是否会违约？\n",
    "\n",
    "> 输入变量可以是离散的，也可以是连续的\n",
    "\n",
    "---\n",
    "\n",
    "#### 二分类\n",
    "\n",
    "我们先从用蓝色圆形数据定义为类型1，其余数据为类型2；只需要分类1次\n",
    "\n",
    "步骤：① ->②\n",
    "\n",
    "<!-- ![image-20210802081155852](http://cdn.ayusummer233.top/img/20210802081202.png) -->\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "#### 多分类\n",
    "\n",
    "我们先定义其中一类为类型1（正类），其余数据为负类（rest）；\n",
    "\n",
    "接下来去掉类型1数据，剩余部分再次进行二分类，分成类型2和负类；\n",
    "\n",
    "如果有𝑛类，那就需要分类 𝑛-1 次\n",
    "\n",
    "步骤：① -> ② -> ③ -> ……\n",
    "\n",
    "<!-- ![image-20210802081340700](http://cdn.ayusummer233.top/img/20210802081340.png) -->\n",
    "\n",
    "----\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Sigmoid 函数\n",
    "\n",
    "𝜎(𝑧) 代表一个常用的逻辑函数（logistic function）, 为𝑆形函数（Sigmoid function）\n",
    "\n",
    "$\\sigma (z) = g(z) = \\frac{1}{1+e^{-z}}$\n",
    "\n",
    "$z=w^Tx+b$\n",
    "\n",
    "合起来，我们得到逻辑回归模型的假设函数：\n",
    "\n",
    "$L(\\hat{y}, y) = -y log\\hat{y} - (1-y) log(1- \\hat{y})$\n",
    "\n",
    "<!-- ![](http://cdn.ayusummer233.top/img/20210802091615.png) -->\n",
    "\n",
    "当 $\\sigma(z) \\geq 0.5$​  时, 预测 $y=1$​\n",
    "\n",
    "当 $\\sigma(z) < 0.5$​​  时, 预测 $y=0$\n",
    "\n",
    "> 注意: 若表达式 $h(x) = z =  w_0 + w_1 x_1 + w_2 x_2 + ... + w_n x_n + b $​, 则 b 可以融入到 $w_0$​, 即: $z=w^Tx$​\n",
    "\n",
    "线性回归的函数 $h(x) = z = w^Tx$​, 范围是 $(−∞, +∞)$。\n",
    "\n",
    "而分类预测结果需要得到 $[0,1]$ 的概率值。\n",
    "\n",
    "在二分类模型中，事件的几率 odds：事件发生与事件不发生的概率之比为 $\\frac{p}{1-p}$, 称为事件的发生比（the odds of experiencing an event）\n",
    "\n",
    "其中𝑝为随机事件发生的概率，𝑝的范围为 $[0,1]$。\n",
    "\n",
    "取对数得到：$log \\frac{p}{1-p}$ 而 $log \\frac{p}{1-p} = w^T x = z$\n",
    "\n",
    "求解得到：$p = \\frac{1}{1+ e^{-w^T x}} = \\frac{1}{1+e^{-z}}$​\n",
    "\n",
    "将 𝑧 进行逻辑变换: $g(z) = \\frac{1}{1+e^{-z}}$\n",
    "\n",
    "$g'(z) = (\\frac{1}{1+e^{-z}})' \\\\ = \\frac{e^{-z}}{(1+e^{-z})^2} \\\\ = \\frac{1 + e^{-z} - 1}{(1+e^{-z})^2} \\\\ = \\frac{1}{(1 + e^{-z})}(1 - \\frac{1}{1 + e^{-z}}) \\\\ = g(z)(1 - g(z))$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n",
    "### 逻辑回归求解\n",
    "\n",
    "假设一个二分类模型： \n",
    "\n",
    "$p(y = 1|x; w) = h(x)$\n",
    "\n",
    "$p(y=0|x;w) = 1-h(x)$\n",
    "\n",
    "则: $p(y|x;w) = (h(x))^y(1-h(x))^{1-y}$​\n",
    "\n",
    "逻辑回归模型的假设是: $h(x)=g(w^Tx)=g(z)$\n",
    "\n",
    "其中 $z = w^Tx$​, 逻辑函数 (logistic function) 公式为:\n",
    "\n",
    "$g(z)=\\frac{1}{1+e^{-z}}, g'(z)=g(z)(1-g(z))$\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "#### 损失函数\n",
    "\n",
    "$L(\\hat{y}, y) = -y log(\\hat{y}) - (1-y)log(1-\\hat{y})$\n",
    "\n",
    "为了衡量算法在全部训练样本上的表现如何, 我们需要定义一个算法的代价函数, 算法的代价函数是对 m 个样本的损失函数求和然后除以 m:\n",
    "\n",
    "---\n",
    "\n",
    "#### 代价函数\n",
    "\n",
    "$J(w) = \\frac{1}{m} \\sum_{i=1}^m L(\\hat{y}^{(i)}, y^{(i)}) = \\frac{1}{m} \\sum_{i=1}^m {( -y^{(i)} log(\\hat{y}^{(i)}) - (1-y^{(i)})log(1-\\hat{y}^{(i)}) )}$\n",
    "\n",
    "---\n",
    "\n",
    "#### 求解过程\n",
    "\n",
    "似然函数为: \n",
    "\n",
    "$L(w) = \\prod_{i=1}^m{P(y^{(i)} | x^{(i)}; w)} = \\prod_{i=1}^m{ (h(x^{(i)}))^{y^{(i)}} (1 - h(x^{(i)}))^{1-y^{(i)}}}$\n",
    "\n",
    "似然函数两边取对数, 则连乘号变成了连加号:\n",
    "\n",
    "$l(w) = log L(w) = \\sum_{i=1}^m{( y^{(i)} log( h(x^{(i)}) ) + (1 - y^{(i)}) log(1 - h(x^{(i)}))    )}$\n",
    "\n",
    "代价函数为:\n",
    "\n",
    "$J(w) = - \\frac{1}{m} l(w) = - \\frac{1}{m} \\sum_{i=1}^m{( y^{(i)} log( h(x^{(i)}) ) + (1 - y^{(i)}) log(1 - h(x^{(i)}))    )}$\n",
    "\n",
    "> [统计学](https://baike.baidu.com/item/统计学/1175)中，似然函数是一种关于[统计模型](https://baike.baidu.com/item/统计模型/7492984)参数的[函数](https://baike.baidu.com/item/函数/301912)。给定输出x时，关于参数θ的似然函数L(θ|x)（在数值上）等于给定参数θ后变量X的概率：L(θ|x)=P(X=x|θ)。\n",
    ">\n",
    "> 似然函数在[推断统计学](https://baike.baidu.com/item/推断统计学/10416457)（Statistical inference）中扮演重要角色，尤其是在参数估计方法中。在教科书中，似然常常被用作“概率”的同义词。但是在统计学中，二者有截然不同的用法。概率描述了已知参数时的随机变量的输出结果；似然则用来描述已知[随机变量](https://baike.baidu.com/item/随机变量/828980)输出结果时，未知参数的可能取值。例如，对于“一枚正反对称的硬币上抛十次”这种事件，我们可以问硬币落地时十次都是正面向上的“概率”是多少；而对于“一枚硬币上抛十次”，我们则可以问，这枚硬币正反面对称的“似然”程度是多少。\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 逻辑回归代码实现\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读取并查看数据源内容\n",
    "从 `iris.data` 中读取几行数据  \n",
    "可以看到它由 4 个特征和 1 个分类标签组成， 类别分类如下：\n",
    "- `Iris-setosa`  山鸢尾\n",
    "- `Iris-versicolor` 变色鸢尾\n",
    "- `Iris-virginica` 弗吉尼亚鸢尾"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.6</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.2</td>\n",
       "      <td>Iris-versicolor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>2.2</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>4.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>5.2</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.1</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.6</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.5</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.5</td>\n",
       "      <td>Iris-versicolor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.7</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>6.8</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1.4</td>\n",
       "      <td>Iris-versicolor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>5.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Iris-versicolor</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width            class\n",
       "92            5.8          2.6           4.0          1.2  Iris-versicolor\n",
       "104           6.5          3.0           5.8          2.2   Iris-virginica\n",
       "24            4.8          3.4           1.9          0.2      Iris-setosa\n",
       "32            5.2          4.1           1.5          0.1      Iris-setosa\n",
       "109           7.2          3.6           6.1          2.5   Iris-virginica\n",
       "22            4.6          3.6           1.0          0.2      Iris-setosa\n",
       "52            6.9          3.1           4.9          1.5  Iris-versicolor\n",
       "123           6.3          2.7           4.9          1.8   Iris-virginica\n",
       "76            6.8          2.8           4.8          1.4  Iris-versicolor\n",
       "60            5.0          2.0           3.5          1.0  Iris-versicolor"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# load the training dataset\n",
    "data = pd.read_csv('iris.data')\n",
    "data.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为三种类别重新命名区分以便后续分类\n",
    "- `Iris-setosa`  0\n",
    "- `Iris-versicolor` 1\n",
    "- `Iris-virginica` 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.4</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>4.4</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>5.6</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>5.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>5.7</td>\n",
       "      <td>2.9</td>\n",
       "      <td>4.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>5.7</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>4.9</td>\n",
       "      <td>2.4</td>\n",
       "      <td>3.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width class\n",
       "80            5.5          2.4           3.8          1.1     1\n",
       "38            4.4          3.0           1.3          0.2     0\n",
       "66            5.6          3.0           4.5          1.5     1\n",
       "110           6.5          3.2           5.1          2.0     2\n",
       "60            5.0          2.0           3.5          1.0     1\n",
       "96            5.7          2.9           4.2          1.3     1\n",
       "99            5.7          2.8           4.1          1.3     1\n",
       "125           7.2          3.2           6.0          1.8     2\n",
       "57            4.9          2.4           3.3          1.0     1\n",
       "0             5.1          3.5           1.4          0.2     0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['class'] = data['class'].str.replace(\"Iris-setosa\", '0')\n",
    "data['class'] = data['class'].str.replace(\"Iris-versicolor\", '1')\n",
    "data['class'] = data['class'].str.replace(\"Iris-virginica\", '2')\n",
    "data.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 分离特征与标签\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris 1 \n",
      "  Features: [5.1, 3.5, 1.4, 0.2] \n",
      "  Label: 0\n",
      "Iris 2 \n",
      "  Features: [4.9, 3.0, 1.4, 0.2] \n",
      "  Label: 0\n",
      "Iris 3 \n",
      "  Features: [4.7, 3.2, 1.3, 0.2] \n",
      "  Label: 0\n",
      "Iris 4 \n",
      "  Features: [4.6, 3.1, 1.5, 0.2] \n",
      "  Label: 0\n"
     ]
    }
   ],
   "source": [
    "# Separate features and labels\n",
    "features = ['sepal_length','sepal_width','petal_length','petal_width']\n",
    "label = 'class'\n",
    "X, y = data[features].values, data[label].values\n",
    "\n",
    "for n in range(0,4):\n",
    "    print(\"Iris\", str(n+1), \"\\n  Features:\",list(X[n]), \"\\n  Label:\", y[n])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 比较特征分布\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAGTCAYAAAAvAHmgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfNklEQVR4nO3de7gkdX3n8fdHBlQugoKOcgnjhbATMShMVBLWHMR4AZVc1IX1EglxRCNedkkcFy+oIQ+u2c2q5BEnIQqrooaIQcclGJ2jkhUiKKg4mkUEYUC5KJeDqIDf/aNqTHOcmdMz3Wf6zPzer+fp53RX/brqW6fmfLrmV9X1S1UhSWrH/SZdgCRpyzL4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BrIpJUksdMuo5JSjKV5LqNzJ/331GSq5M8bT7XoYXH4G9c/4d/V5KZJD9KsirJPpOua50kL01y4aTrkLYlBr8AnlNVOwOPAH4AvGfC9cybJIsmXYM0aQa/fqGqfgKcA/zaumlJdk1yVpKbklyT5I1J7pfkIUmuS/Kcvt3OSa5M8pL+9QeSnJ7kM0nuSPL5JPuub70bWcdS4HTgkP5/JLdu4P2PTPKFfj3/nOSvk3ywn7ek7zI5Lsn3gM/1y35jv64b+3Xv2rf/pe6Xwe6QJCcnOSfJR/v1fSXJgQNt90zyD/22fDfJqwfmPbD/vfwoyTeB3xhitxyR5KokNyd5Z1/7Dkl+mORxA8t+WJIfJ3noBn5HL0uypq/5m0kOWk+bJyb5UpJbk9yQ5LQkO/TzkuSv+t/X7Um+nuSAft4R/TLvSLI2yYlDbJcmyODXLyTZEfhPwEUDk98D7Ao8Cvht4CXAsVX1Q+CPgL9J8jDgr4DLquqsgfe+EHg7sAdwGfChDax6Q+tYAxwPfKmqdq6q3Tbw/g8D/wrsDpwMvHg9bX4bWAo8A3hp/zisX+fOwGkbWPb6HAX8PfCQft2fSLJ9kvsBnwQuB/YCDgdem+QZ/fveAjy6fzwD+MMh1vV7wDLgoH69f1RVPwM+ArxooN0xwGer6qbZC0jyfLrfy0uABwHPBW5Zz7ruBV5Ht78O6et/ZT/v6cBTgF+l21cvGFjGGcDLq2oX4ADgc0Nslyapqnw0/ACuBmaAW4G7geuBx/XztgN+BvzaQPuXA9MDr98DfB1YC+w+MP0DwEcGXu9MFyz79K8LeMxc66AL6As3Uv+vAPcAOw5M+yDwwf75kn5djxqY/1nglQOv9++3fREwBVy3nt/R0/rnJwMXDcy7H3AD8B+BJwHfm/XeNwDv759fBTxzYN7y2eua9d6a1f6VdOHOunUB6V9fArxgA8v5J+A1G9n/T9vAvNcC5/bPnwr8G/Bk4H6z2n2v32cPmvS/Zx/DPTziF8DvVnc0/QDgVcDnkzyc7shve+CagbbX0B3NrrOS7ijvA1U1+yjy2nVPqmoG+CGw56w2w6xjY/YEflhVP17fejcwbc/1rG8RsHjIdQ5u18+B6/pl7gvs2XeV3Np3Tf23geXuOauOwRrmXFfffs9+vRcDPwamkvwHug/R8zawjH2A78y1oiS/muRTSb6f5HbgL+j2D1X1Obr/Ff01cGOSlUke1L/1D4AjgGv6Lr1DhtguTZDBr1+oqnur6uN0R+aHAjfTHQkP9s3/Ct3RPUm2owv+s4BX5pcvPfzF1UFJdqbrGrl+VpuNroPuqHdjbgAe0ndT/dJ6Bzdv4Pn161nfPXQntu8EfrGsfhtn95sPbtf9gL37ZV4LfLeqdht47FJVRwzUOljbr8yxbbO35Ve47+/vTLrunhcD51R3jmZ9rqXrXprLe4FvAftV1YPoPrSybmZVvbuqDqY7B/SrwJ/2079cVUcBDwM+AXxsiHVpggx+/UJ/Au8o4MHAmqq6l+6P+JQku/QnZ/8LXVcKdMFQdH397wTO6oNynSOSHNqfIHw7XRfJfY7Gh1jHD4C9151knK2qrqHr5ji5P+l5CPCcOTb1bOB1/UnhnemObD9aVffQdWc8IMmRSbYH3gjcf9b7D07y++muEHot8FO68yL/CtyR5PX9idztkhyQZN1J3I8Bb0jy4CR7AyfMUSfAn/bt9wFeA3x0YN4H6c4BvIjuw3dD/hY4McnB/T5+TNZ/on0X4HZgpv9fxCvWzUjyG0me1P9O7gR+Avy8/52/MMmuVXV3//6fD7FdmqRJ9zX5mOyDro/3Lrp+/juAbwAvHJj/YLqAuYnuyPHNdAcMBwM/Ah7Tt9sO+BfgpP71B+iuyPlMv+wvAI8cWG4NvHe96+jn7QCsousmunkD2/Bo4It9/Z+l+1/IGf28Jf26Fg20v1+/jmv7dX4QePDA/JfSHZ3fCJzIL/fxn0MXwHcAXwUOGnjvnnQfLN/vfz8XDbx3R7qAvhX4Jt0R81x9/K+mOzdwC/A/gO1mtfnnvr7MsZ+PB77d74tvAE8Y2P/r6nsK3RH/TP/7fBv9+RW6E71f6+fdTHeifud+/5zfb+vtwJeBQyf979rHxh/rTgxJY5XkA3Sh9sYJrPujwLeq6i3zsOyT6T6wXjRX2y0hyd8B10/i96ytl19m0Vav70r5IfBdussOjwJOnWhRW0CSJcDvA0+YcCnaytjHr23Bw4Fpum6IdwOvqKqvTrSieZbk7XRdNu+squ9Ouh5tXezqkaTGeMQvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg19iuGEO+3vp//mWqmnWuh0iUWNj8EsLzCQ/YNQGg1+SGmPwa0Hq73C5th/O79tJDu+HHVyR5DtJbknysSQP6duvG2JxeZLr+6EDTxxY3gaHFRyhxmcnuaxf5v9N8usD865OcmKSryW5Ld1QjQ8YmP9nfR3XJ/njdV1NSZbTjVz2Z+mGm/zkwCofv6HlSZvC4NeCk2R/ugFhfqO64fyeQXcXyROA36UbRnFPujtC/vWstx8G7Ed3z57XD/SLb2xYwc2p8QnA39GNPLU78D7gvCSDt3B+AfBM4JHAr9Pd9ZMkz6S79fTT6AZQmVr3hqpaSXfny/9e3XCTz5lredKmMvi1EN1Ldw/8X0uyfVVdXVXfobu18ElVdV1V/ZTuFsnP6++Lv85bq+rOqvo68H66sWipqkur6qKquqeqrqYL6t8eocblwPuq6uLqBrA5k+6+/E8eaPPuqrq+uvGJPwk8vp/+ArrhGK+obuSwk4dc54aWJ20Sg18LTlVdSTfAycl0w/x9JMm6oQ3Pzb8Pa7iG7kNicMjE9Q5VmI0MK7iZ9gX+a+47zOI+3Hdoye8PPP8x3f3r4ZeHYFzfUJHrs6HlSZvE4NeCVFUfrqpD6QK2gHfQBeSz6r5DGz6gqtYOvHVDQxVudFjBzXAtcMqsWnasqrOHeO8NdMM1rq9mmHu4SWkkBr8WnCT7J3lq31/+E7oRwn5ON6LXKeuGDUzy0HRDRQ56U5IdkzwWOJZ/H6pwg8MKbqa/AY7vhyNMkp364Rp3GeK9HwOOTbI03VjBb5o1/wfAo0asT9ogg18L0f3pBlK5ma5742HAG4B3AecBFyS5g25YwyfNeu/ngSvphmD8y6q6oJ9+IvCf6YZL/BvuO3btJquqS4CXAafRnWS+kiFPtlbV/6EbN2B1/76L+lk/7X+eQXd+49YknxilTml9vB+/tgn9aFTfBbavbtD0rUaSpXSDqtx/a6tdWyeP+KUJSPJ7Se6f5MF05y8+aehrSzH4pQFJrui/ODX78cIxr+rlwI3Ad+iuTBr1nIM0NLt6JKkxHvFLUmMWzd1kfuyxxx61ZMmSSa1+i7nzzjvZaaedJl2GxsB9ue1oZV9eeumlN1fVQ2dPn1jwL1myhEsuuWRSq99ipqenmZqamnQZGgP35bajlX2Z5Jr1TberR5IaM1TwJ3ldf7XDN5KcPft2sP1laR9NcmWSi/trqiVJC9CcwZ9kL+DVwLKqOgDYDjh6VrPjgB9V1WOAv6K7LlmStAAN29WzCHhgf/vbHfn3G1+tcxRwZv/8HODwJKPcAEuSNE/mPLlbVWuT/CXwPbqbZV0wcP+Tdfaiv7VsVd2T5Da6wSluHmzUjy60HGDx4sVMT0+PvAEL3czMTBPb2QL35baj9X05Z/D3Xyk/im7Un1uBv0/yoqr64KaurB9daCXAsmXLqoWz6q1cPdAC9+W2o/V9OUxXz9OA71bVTVV1N/Bx4DdntVlLf0/xvjtoV+CWcRYqSRqPYYL/e8CT+3uch26s0jWz2pwH/GH//HnA58p7QUjSgjRn8FfVxXQnbL8CfL1/z8okb0vy3L7ZGcDuSa6kG0R6xTzVK0ka0VDf3K2qtwBvmTX5zQPzfwI8f4x1SZLmid/claTGGPyS1BiDX5IaM7G7c0rSfBnXjQO21YsTPeKXtM2pqo0+9n39p+Zss62GPhj8ktQcg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1Jj5gz+JPsnuWzgcXuS185qM5XktoE2b563iiVJI1k0V4Oq+jbweIAk2wFrgXPX0/SLVfXssVYnSRq7Te3qORz4TlVdMx/FSJLm35xH/LMcDZy9gXmHJLkcuB44saqumN0gyXJgOcDixYuZnp7exNVvfWZmZprYzha4L7ctLe/LVNVwDZMd6EL9sVX1g1nzHgT8vKpmkhwBvKuq9tvY8pYtW1aXXHLJZpa99ZienmZqamrSZWgM3JfbjiUrVnH1qUdOuox5l+TSqlo2e/qmdPU8C/jK7NAHqKrbq2qmf/5pYPske2x2tZKkebMpwX8MG+jmSfLwJOmfP7Ff7i2jlydJGreh+viT7AT8DvDygWnHA1TV6cDzgFckuQe4Czi6hu1DkiRtUUMFf1XdCew+a9rpA89PA04bb2mSpPngN3clqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjZkz+JPsn+SygcftSV47q02SvDvJlUm+luSgeatYkjSSRXM1qKpvA48HSLIdsBY4d1azZwH79Y8nAe/tf0qSFphN7eo5HPhOVV0za/pRwFnVuQjYLckjxlKhJGms5jzin+Vo4Oz1TN8LuHbg9XX9tBsGGyVZDiwHWLx4MdPT05u4+q3PzMxME9u5tTvssMPGtqzVq1ePbVmaPy3/XQ4d/El2AJ4LvGFzV1ZVK4GVAMuWLaupqanNXdRWY3p6mha2c2tXVXO2WbJiFVefeuQWqEbz7vxVTf9dbkpXz7OAr1TVD9Yzby2wz8DrvftpkqQFZlOC/xjW380DcB7wkv7qnicDt1XVDRtoK0maoKG6epLsBPwO8PKBaccDVNXpwKeBI4ArgR8Dx469UknSWAwV/FV1J7D7rGmnDzwv4E/GW5okaT74zV1JaozBL0mN2dTr+CVpog586wXcdtfdIy9nyYpVI71/1wduz+VvefrIdUyCwS9pq3LbXXeP/H2KcXy/ZtQPjkmyq0eSGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNGSr4k+yW5Jwk30qyJskhs+ZPJbktyWX9483zU64kaVSLhmz3LuD8qnpekh2AHdfT5otV9ezxlSZJmg9zBn+SXYGnAC8FqKqfAT+b37IkSfNlmCP+RwI3Ae9PciBwKfCaqrpzVrtDklwOXA+cWFVXzF5QkuXAcoDFixczPT09Su1bhZmZmSa2sxXuy8nbZekKHnfmitEXdOaodcD09E6j1zEJVbXRB7AMuAd4Uv/6XcDbZ7V5ELBz//wI4P/NtdyDDz64WrB69epJl6Ax2ff1n5p0Carx7Idx/F1uDf8egEtqPfk7zMnd64Drquri/vU5wEGzPjxur6qZ/vmnge2T7DHiZ5IkaR7MGfxV9X3g2iT795MOB7452CbJw5Okf/7Efrm3jLlWSdIYDHtVzwnAh/oreq4Cjk1yPEBVnQ48D3hFknuAu4Cj+/9mSJIWmKGCv6ouo+vrH3T6wPzTgNPGV5Ykab74zV1JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjRn2Xj3SVu3At17AbXfdPfJylqxYNdL7d33g9lz+lqePXIc0CoNfTbjtrru5+tQjR1rG9PQ0U1NTIy1j1A8OaRzs6pGkxhj8ktQYg1+SGmPwS1JjDH5JaoxX9Uja6ozl6qjzR780d2tl8Evaqox6WS50HxzjWM7Wyq4eSWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4JekxgwV/El2S3JOkm8lWZPkkFnzk+TdSa5M8rUkB81PuZKkUQ17W+Z3AedX1fOS7ADsOGv+s4D9+seTgPf2PyVJC8ycR/xJdgWeApwBUFU/q6pbZzU7CjirOhcBuyV5xLiLlSSNbpgj/kcCNwHvT3IgcCnwmqq6c6DNXsC1A6+v66fdMLigJMuB5QCLFy9menp68yvfSszMzDSxnVuDUffDuPal/x4Whpb3wzDBvwg4CDihqi5O8i5gBfCmTV1ZVa0EVgIsW7aspqamNnURW53p6Wla2M4F7/xVI++HsezLMdShMWh8Pwxzcvc64Lqqurh/fQ7dB8GgtcA+A6/37qdJkhaYOYO/qr4PXJtk/37S4cA3ZzU7D3hJf3XPk4HbquoGJEkLzrBX9ZwAfKi/oucq4NgkxwNU1enAp4EjgCuBHwPHzkOtkqQxGCr4q+oyYNmsyacPzC/gT8ZXliRpvvjNXUlqjMEvSY0Zto9f2qrtsnQFjztzxegLOnPUOgCOHL0OaQQGv5pwx5pTufrU0QJ3HNfxL1mxaqT3S+NgV48kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjJdzjiDJWJbT3fFCkrYMj/hHUFVzPvZ9/afmbCNJW5LBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMciEXNWLJi1egLOX+0Zez6wO1Hr0FzGmaQpLxj7uVsq+NlGPxqwtWnHjnyMpasWDWW5Wj+zRXY09PTTE1NbZliFiC7eiSpMQa/JDXG4JekxgzVx5/kauAO4F7gnqpaNmv+FPCPwHf7SR+vqreNrUpJ0thsysndw6rq5o3M/2JVPXvUgiRJ88uuHklqzLBH/AVckKSA91XVyvW0OSTJ5cD1wIlVdcXsBkmWA8sBFi9ezPT09OZVvZVpZTtb4L7cNszMzDS9L4cN/kOram2ShwGfSfKtqvrCwPyvAPtW1UySI4BPAPvNXkj/gbESYNmyZdXEdbTnr2r6euFtivtym+F1/EOoqrX9zxuBc4Enzpp/e1XN9M8/DWyfZI8x1ypJGoM5gz/JTkl2WfcceDrwjVltHp7+O9JJntgv95bxlytJGtUwXT2LgXP7XF8EfLiqzk9yPEBVnQ48D3hFknuAu4Cja1u9yYUkbeXmDP6qugo4cD3TTx94fhpw2nhLkyTNBy/nlKTGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUmGHG3G3WgW+9gNvuunvk5SxZsWqk9+/6wO25/C1PH7kOSQKDf6Nuu+turj71yJGWMT09zdTU1EjLGPWDQ5IG2dUjSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYMFfxJrk7y9SSXJblkPfOT5N1JrkzytSQHjb9USdI4bMrdOQ+rqps3MO9ZwH7940nAe/ufkqQFZlxdPUcBZ1XnImC3JI8Y07IlSWM07BF/ARckKeB9VbVy1vy9gGsHXl/XT7thsFGS5cBygMWLFzM9Pb05NW8xuyxdwePOXDH6gs4ctQ6Ynt5p9Do0soX+b1bDmZmZaXpfDhv8h1bV2iQPAz6T5FtV9YVNXVn/gbESYNmyZTXqACXz7Y4Vpy6YgVim/nC0ZWgMzl818r7UwjCOv8ut2VBdPVW1tv95I3Au8MRZTdYC+wy83rufJklaYOYM/iQ7Jdll3XPg6cA3ZjU7D3hJf3XPk4HbquoGJEkLzjBdPYuBc5Osa//hqjo/yfEAVXU68GngCOBK4MfAsfNTriRpVHMe8VfVVVV1YP94bFWd0k8/vQ99+qt5/qSqHl1Vj6uqX7rWX5Im7eyzz+aAAw7g8MMP54ADDuDss8+edEkTsSnX8UvSVuvss8/mpJNO4owzzuDee+9lu+2247jjjgPgmGOOmXB1W5a3bJDUhFNOOYUzzjiDww47jEWLFnHYYYdxxhlncMopp0y6tC3OI34J6M9hzd3uHXO3qaoRq9F8WLNmDYceeuh9ph166KGsWbNmQhVNjkf8El1Yz/VYvXr1UO20MC1dupQLL7zwPtMuvPBCli5dOqGKJsfgl9SEk046ieOOO47Vq1dzzz33sHr1ao477jhOOumkSZe2xdnVI6kJ607gnnDCCaxZs4alS5dyyimnNHdiFwx+SQ055phjOOaYY7xlw6QLkCRtWQa/JDXG4JekxtjHP4clK1aNvpDzR1vGrg/cfvQaJKln8G/EqPfih+6DYxzLkaRxsatHkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNGTr4k2yX5KtJPrWeeS9NclOSy/rHH4+3TEnSuCzahLavAdYAD9rA/I9W1atGL0mSNJ+GOuJPsjdwJPC381uOJGm+DXvE/7+APwN22UibP0jyFODfgNdV1bWzGyRZDiwHWLx4MdPT05tU7Naqle3c1s3MzLgvtxGt78s5gz/Js4Ebq+rSJFMbaPZJ4Oyq+mmSlwNnAk+d3aiqVgIrAZYtW1ZTUxta3Dbk/FU0sZ0NmJ6edl9uI1rfl8N09fwW8NwkVwMfAZ6a5IODDarqlqr6af/yb4GDx1qlJGls5gz+qnpDVe1dVUuAo4HPVdWLBtskecTAy+fSnQSWJC1Am3JVz30keRtwSVWdB7w6yXOBe4AfAi8dT3mSpHHbpOCvqmlgun/+5oHpbwDeMM7CJEnzw2/uSlJjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWrMZt+rR5BkuHbv2Pj8qhpDNZI0HI/4R1BVcz5Wr149ZxtJ2pIMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMZkUneHTHITcM1EVr5l7QHcPOkiNBbuy21HK/ty36p66OyJEwv+ViS5pKqWTboOjc59ue1ofV/a1SNJjTH4JakxBv/8WznpAjQ27sttR9P70j5+SWqMR/yS1BiDX5IaY/DPkyTPTPLtJFcmWTHperT5kvxdkhuTfGPStWg0SfZJsjrJN5NckeQ1k65pEuzjnwdJtgP+Dfgd4Drgy8AxVfXNiRamzZLkKcAMcFZVHTDperT5kjwCeERVfSXJLsClwO+29rfpEf/8eCJwZVVdVVU/Az4CHDXhmrSZquoLwA8nXYdGV1U3VNVX+ud3AGuAvSZb1ZZn8M+PvYBrB15fR4P/uKSFLMkS4AnAxRMuZYsz+CU1J8nOwD8Ar62q2yddz5Zm8M+PtcA+A6/37qdJmrAk29OF/oeq6uOTrmcSDP758WVgvySPTLIDcDRw3oRrkpqXJMAZwJqq+p+TrmdSDP55UFX3AK8C/onu5NHHquqKyValzZXkbOBLwP5Jrkty3KRr0mb7LeDFwFOTXNY/jph0UVual3NKUmM84pekxhj8ktQYg1+SGmPwS1JjDH5JaozBLw1IcnKSEyddhzSfDH5JaozBr6YleUmSryW5PMn/njXvZUm+3M/7hyQ79tOfn+Qb/fQv9NMem+Rf+y8EfS3JfpPYHmkYfoFLzUryWOBc4Der6uYkDwFeDcxU1V8m2b2qbunb/jnwg6p6T5KvA8+sqrVJdquqW5O8B7ioqj7U36Zju6q6a1LbJm2MR/xq2VOBv6+qmwGqavY99w9I8sU+6F8IPLaf/i/AB5K8DNiun/Yl4L8leT2wr6GvhczglzbsA8CrqupxwFuBBwBU1fHAG+nuwHpp/z+DDwPPBe4CPp3kqZMpWZqbwa+WfQ54fpLdAfqunkG7ADf0t/F94bqJSR5dVRdX1ZuBm4B9kjwKuKqq3g38I/DrW2QLpM2waNIFSJNSVVckOQX4fJJ7ga8CVw80eRPd6Ew39T936ae/sz95G+CzwOXA64EXJ7kb+D7wF1tkI6TN4MldSWqMXT2S1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXm/wMS+L11ywRSEwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAGTCAYAAAAvAHmgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdnklEQVR4nO3dfbQddX3v8feHEAkKgoqmPEm0WG8Qa5G01itXE7DVgoXeqr1yfagtBakVRUpLAK+ivfSSZS+2oqtcMK34UIT60FJoUas5KF2KAoIKsRYVBaTyJA9BSA393j9mgpvjOTkn2ftkJ/m9X2vtlb1nfjPznT05nzPnN3vPL1WFJKkd2427AEnS5mXwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuDXWCSpJPuOu45xSrI0yc0bmD/n71GSG5O8cC63oS2Pwd+4/gf/gSRrkvwwySVJ9h53XesleW2Sy8ddh7QtMfgF8OtVtROwO/AD4Kwx1zNnkmw/7hqkcTP49bCqehD4KLDf+mlJdknygSS3J/lukrck2S7J45PcnOTX+3Y7JbkhyWv61+9PcnaSTye5L8llSfaZarsb2MZi4Gzguf1fJHdPs/xTknyu384/J3lvkg/18xb1XSZHJfke8Nl+3W/pt3Vbv+1d+vY/1f0y2B2S5LQkH01yQb+9q5M8a6DtHkk+1u/Ld5K8cWDejv378sMk1wO/OIvDcmiSbye5I8k7+9ofleSuJM8cWPeTkvwoyROneY+OTrK6r/n6JM+eos0vJflCkruT3JrkPUke1c9Lknf179e9Sb6WZP9+3qH9Ou9LckuSE2exXxojg18PS/Jo4H8AXxyYfBawC/BU4AXAa4Dfqaq7gN8Fzk3yJOBdwDVV9YGBZV8J/AmwG3AN8OFpNj3dNlYDxwJfqKqdqmrXaZb/G+BLwBOA04BXT9HmBcBi4EXAa/vHsn6bOwHvmWbdUzkC+Fvg8f22/y7J/CTbAf8AXAvsCRwCHJ/kRf1ybwN+tn+8CPjtWWzrvwNLgGf32/3dqvoP4CPAqwbaHQl8pqpun7yCJC+ne19eAzwWOBy4c4ptPQS8me54Pbev//X9vF8Fng/8HN2x+q2BdawEXldVOwP7A5+dxX5pnKrKR8MP4EZgDXA38GPg+8Az+3nzgP8A9hto/zpgYuD1WcDXgFuAJwxMfz/wkYHXO9EFy9796wL2nWkbdAF9+QbqfzKwDnj0wLQPAR/qny/qt/XUgfmfAV4/8Prp/b5vDywFbp7iPXph//w04IsD87YDbgX+G/Ac4HuTlj0Z+Ov++beBFw/MO2bytiYtW5Pav54u3Fm/LSD96yuB35pmPZ8E3rSB4//CaeYdD3yif34w8E3gl4HtJrX7Xn/MHjvu/88+ZvfwjF8Av1Hd2fQC4A3AZUl+hu7Mbz7w3YG236U7m13vHLqzvPdX1eSzyJvWP6mqNcBdwB6T2sxmGxuyB3BXVf1oqu1OM22PKba3PbBwltsc3K//BG7u17kPsEffVXJ33zV1ysB695hUx2ANM26rb79Hv90rgB8BS5P8F7pfohdNs469gW/NtKEkP5fk4iT/nuRe4E/pjg9V9Vm6v4reC9yW5Jwkj+0XfSlwKPDdvkvvubPYL42Rwa+HVdVDVfVxujPzg4A76M6EB/vmn0x3dk+SeXTB/wHg9fnpjx4+/OmgJDvRdY18f1KbDW6D7qx3Q24FHt93U/3Udgd3b+D596fY3jq6C9v3Aw+vq9/Hyf3mg/u1HbBXv86bgO9U1a4Dj52r6tCBWgdre/IM+zZ5X57MI9+/8+i6e14NfLS6azRTuYmue2kmfwl8A3haVT2W7pdW1s+sqndX1YF014B+DvijfvqXq+oI4EnA3wEXzmJbGiODXw/rL+AdATwOWF1VD9H9EJ+eZOf+4uwJdF0p0AVD0fX1vxP4QB+U6x2a5KD+AuGf0HWRPOJsfBbb+AGw1/qLjJNV1XfpujlO6y96Phf49Rl29Xzgzf1F4Z3ozmwvqKp1dN0ZC5IclmQ+8BZgh0nLH5jkN9N9Quh4YC3ddZEvAfclOam/kDsvyf5J1l/EvRA4OcnjkuwFHDdDnQB/1LffG3gTcMHAvA/RXQN4Fd0v3+m8DzgxyYH9Md43U19o3xm4F1jT/xXx++tnJPnFJM/p35P7gQeB/+zf81cm2aWqftwv/5+z2C+N07j7mnyM90HXx/sAXT//fcDXgVcOzH8cXcDcTnfm+Fa6E4YDgR8C+/bt5gH/Apzav34/3SdyPt2v+3PAUwbWWwPLTrmNft6jgEvouonumGYffhb4fF//Z+j+ClnZz1vUb2v7gfbb9du4qd/mh4DHDcx/Ld3Z+W3Aifx0H/9H6QL4PuArwLMHlt2D7hfLv/fvzxcHln00XUDfDVxPd8Y8Ux//G+muDdwJ/F9g3qQ2/9zXlxmO87HAv/bH4uvAAQPHf319z6c741/Tv5/voL++Qneh96v9vDvoLtTv1B+fS/t9vRf4MnDQuP9f+9jwY/2FIWmkkryfLtTeMoZtXwB8o6reNgfrPo3uF9arZmq7OST5K+D743iftfXyyyza6vVdKXcB36H72OERwBljLWozSLII+E3ggDGXoq2MffzaFvwMMEHXDfFu4Per6itjrWiOJfkTui6bd1bVd8Zdj7YudvVIUmM845ekxhj8ktQYg1+SGmPwS1JjDH5JaozBL02SEQ55mOSfkkx5++X8ZKyAab9PM8papPX8Apc0h6rq12bbNskE3e2k3zd3FUme8UtScwx+bfH6u13e0g/t969JDkk3BOHyJN9KcmeSC5M8vm+/vgvlmCTf74cRPHFgfdMOMTjLep7SL7td//rcJLcNzP9gkuP75xNJfq9/Pi/Jn6UbRvHbwGEDy5xON5jLe9INMzk4ItgLk/xbv833JgnSEAx+bdGSPJ1ucJhfrG5ovxfR3VHyOOA36IZU3IPu7pDvnbT4MuBpdPfvOSn9uLlseIjBGfW3SLiXn9wj5/l0tzJe3L9+AXDZFIseDbykX24J8LKBdZ5Kd0fMN1Q3zOQbBpZ7Cd34vD9PN+Thi5CGYPBrS/cQ3f3w90syv6purKpv0d1m+NSqurmq1tLdLvllky6Uvr2q7q+qrwF/TTcuLVV1VVV9sarWVdWNwP+jC+uNcRnwgnQjlUF3q+YXJHkK3bi2106xzG8Bf15VN1U3ZvH/meW2zqiqu6vqe8Aq4Bc2slbpEby4qy1aVd3Qd5ucBjwjySfpBmrZB/hEksFBPx7ikcMnTh628JnQDTEInEl31v1oup+DqzaytMvoBi2/mW6sgQm6kbAeBD5f3ZCMk23K0IvQ3dt/vR/R3Qdf2mSe8WuLV1V/U1UH0YV9ASvoAvTX6pHDHC6oqlsGFp1u2MINDjE4S5fR9ckv7Z9fDjyP6bt5YOahF71jojYLg19btCRPT3Jwkh3ozqYfoBva72y64Rr36ds9sR82ctD/SvLoJM8AfoefDFs47RCDs1VV/9bX8irgsqq6l26YyJcyffBfCLwxyV5JHgcsnzT/B8BTN7YWaWMZ/NrS7UA3qModdF0eTwJOBv4CuAj4VJL76IY4fM6kZS8DbqAbjvHPqupT/fQTgf9JN3TiuTxyHNuNcRlwZ/1kHOHL6P5yuHqa9ucCn6Tr/78a+Pik+X9Bd53ih0nevYk1STPyfvza5vQjU30HmF/dAOqSBnjGL0mNMfilaSS5rv8y1eTHK8ddmzQMu3okqTGzPuPvv27+lSQXTzHvtUluT3JN//i90ZYpSRqVjfkC15uA1XTfSpzKBZO+Zr5Bu+22Wy1atGgjNr91uv/++3nMYx4z7jI0Ah7LbUcrx/Kqq666o6qeOHn6rII/yV50N5Q6ne5bk0NbtGgRV1555ShWtUWbmJhg6dKl4y5DI+Cx3Ha0ciyTTPnt8Nme8f858Md0X3yZzkuTPB/4JvDmgc82DxZxDHAMwMKFC5mYmJjl5rdea9asaWI/W+Cx3Ha0fixnDP4kLwFuq6qrkiydptk/AOdX1dokrwPOAw6e3KiqzgHOAViyZEm18Bu3lTOLFngstx2tH8vZXNx9HnB4khuBjwAHJ/nQYIOqurO/QyLA+4ADR1qlJGlkZgz+qjq5qvaqqkXAK4DPVtWrBtsk2X3g5eF0F4ElSVugTb4tc5J3AFdW1UV0N546HFgH3AW8djTlSZJGbaOCv6om6O47TlW9dWD6yXQ3zpIkbeG8ZYMkNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY3Z5C9wCZKMZD0OhiNpc/KMfwhVNeNjn5MunrGNJG1OBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhoz6+BPMi/JV5JcPMW8HZJckOSGJFckWTTSKiVJI7MxZ/xvAlZPM+8o4IdVtS/wLmDFsIVJkubGrII/yV7AYcD7pmlyBHBe//yjwCFJMnx5kqRR236W7f4c+GNg52nm7wncBFBV65LcAzwBuGOwUZJjgGMAFi5cyMTExMZXvBVqZT+3dWvWrPFYbiNaP5YzBn+SlwC3VdVVSZYOs7GqOgc4B2DJkiW1dOlQq9s6XHoJTexnAyYmJjyW24jWj+VsunqeBxye5EbgI8DBST40qc0twN4ASbYHdgHuHGGdkqQRmTH4q+rkqtqrqhYBrwA+W1WvmtTsIuC3++cv69vUSCuVJI3EbPv4f0qSdwBXVtVFwErgg0luAO6i+wUhSdoCbVTwV9UEMNE/f+vA9AeBl4+yMEnS3PCbu5LUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4JfUjOOOO44FCxawbNkyFixYwHHHHTfuksZik0fgkqStyXHHHcfZZ5/NihUr2G+//bj++us56aSTADjrrLPGXN3m5Rm/pCace+65rFixghNOOIEFCxZwwgknsGLFCs4999xxl7bZGfySmrB27VqOPfbYR0w79thjWbt27ZgqGh+DX1ITdthhB84+++xHTDv77LPZYYcdxlTR+NjHL6kJRx999MN9+vvttx9nnnkmJ5100k/9FdACg19SE9ZfwD3llFNYu3YtO+ywA8cee2xzF3bBrh5JDTnrrLN48MEHWbVqFQ8++GCToQ8GvyQ1x+CXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1Bhv0rYBz3r7p7jngR8PvZ5Fyy8ZavlddpzPtW/71aHrkCQw+Dfongd+zI1nHDbUOiYmJli6dOlQ6xj2F4ckDbKrR5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxvg5fknbnCQjWU9VjWQ9WxrP+CVtc6pqg499Trp4xjbbauiDwS9JzTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmNmDP4kC5J8Kcm1Sa5L8vYp2rw2ye1Jrukfvzc35UqShjWbb+6uBQ6uqjVJ5gOXJ/mnqvripHYXVNUbRl+iJGmUZgz+6r6+tqZ/Ob9/bLtfaZOkbdys7tWTZB5wFbAv8N6qumKKZi9N8nzgm8Cbq+qmKdZzDHAMwMKFC5mYmNjUujebYWtcs2bNSPZza3ivtnWjOpbaMrR8LGcV/FX1EPALSXYFPpFk/6r6+kCTfwDOr6q1SV4HnAccPMV6zgHOAViyZEkNOwj5nLv0kqEHSh/FYOujqEPDG8mx1Jah8Z+pjfpUT1XdDawCXjxp+p1VtbZ/+T7gwJFUJ0kaudl8queJ/Zk+SXYEfgX4xqQ2uw+8PBxYPcIaJUkjNJuunt2B8/p+/u2AC6vq4iTvAK6sqouANyY5HFgH3AW8dq4KliQNZzaf6vkqcMAU09868Pxk4OTRliZJmgt+c1eSGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1Zla3bGjVzouX88zzlg+/ovOGrQPgsOHr0LSSjGxd3X0NpS2Xwb8B960+gxvPGC5wR3F/l0XLLxlqec1sNmG9aPklQ/9/kLYEdvVIUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqzPbjLkCSNsaz3v4p7nngx0OvZ9HyS4Zafpcd53Pt23516DrGweCXtFW554Efc+MZhw21jomJCZYuXTrUOob9xTFOdvVIUmMMfklqjMEvSY0x+CWpMV7cncFILuBcOvynByRpVAz+DRj2kwPQ/eIYxXokaVTs6pGkxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUmBmDP8mCJF9Kcm2S65K8fYo2OyS5IMkNSa5IsmhOqpUkDW02Z/xrgYOr6lnALwAvTvLLk9ocBfywqvYF3gWsGGmVkqSRmTH4q7Omfzm/f9SkZkcA5/XPPwockiQjq1KSNDKzujtnknnAVcC+wHur6opJTfYEbgKoqnVJ7gGeANwxaT3HAMcALFy4kImJiaGK31q0sp9bsj/4zP3cP/z43EPfpvsx8+G9hzxm+EIatvPi5TzzvOXDr+i8mZtsuA6YmNhKj2VVzfoB7AqsAvafNP3rwF4Dr78F7LahdR144IHVgn1OunjcJahGcxxWrVq1RdTROo/l7AFX1hT5u1Gf6qmqu/vgf/GkWbcAewMk2R7YBbhzE38XSZLm0Gw+1fPEJLv2z3cEfgX4xqRmFwG/3T9/GfDZ/reNJGkLM5s+/t2B8/p+/u2AC6vq4iTvoPsz4iJgJfDBJDcAdwGvmLOKJUlDmTH4q+qrwAFTTH/rwPMHgZePtjRJ0lzwm7uS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSY2Yz2Lq01dt58XKeed7y4Vd03rB1ABw2fB2NW7T8kuFXculw69hlx/nD1zAmBr+acN/qM7jxjOECd2JigqVLlw61jpEEVuOGPY7QHYdRrGdrZVePJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTHbz9Qgyd7AB4CFQAHnVNVfTGqzFPh74Dv9pI9X1TtGWqk0pEXLLxl+JZcOt45ddpw/fA3SkGYMfmAd8IdVdXWSnYGrkny6qq6f1O7zVfWS0ZcoDe/GMw4beh2Lll8ykvVI4zZjV09V3VpVV/fP7wNWA3vOdWGSpLkxmzP+hyVZBBwAXDHF7OcmuRb4PnBiVV03xfLHAMcALFy4kImJiY2td6vUyn62wGO5dVi2bNmMbbJi5vWsWrVqBNVseWYd/El2Aj4GHF9V906afTWwT1WtSXIo8HfA0yavo6rOAc4BWLJkSS1dunQTy96KXHoJTexnCzyWW42q2uD8iYmJpo/lrD7Vk2Q+Xeh/uKo+Pnl+Vd1bVWv65/8IzE+y20grlSSNxIzBnyTASmB1VZ05TZuf6duR5Jf69d45ykIlSaMxm66e5wGvBr6W5Jp+2inAkwGq6mzgZcDvJ1kHPAC8omb6W0uSNBYzBn9VXQ5khjbvAd4zqqIkSXPHb+5KUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWrMbMbc1TT68eVnbrdiw/MdnljS5uQZ/xCqasbHqlWrZmwjSZuTwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvqRnnn38++++/P4cccgj7778/559//rhLGgtv2SCpCeeffz6nnnoqK1eu5KGHHmLevHkcddRRABx55JFjrm7z8oxfUhNOP/10Vq5cybJly9h+++1ZtmwZK1eu5PTTTx93aZudwS+pCatXr+aggw56xLSDDjqI1atXj6mi8TH4JTVh8eLFXH755Y+Ydvnll7N48eIxVTQ+Br+kJpx66qkcddRRrFq1inXr1rFq1SqOOuooTj311HGXttl5cVdSE9ZfwD3uuONYvXo1ixcv5vTTT2/uwi4Y/JIacuSRR3LkkUcyMTHB0qVLx13O2NjVI0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqzIzBn2TvJKuSXJ/kuiRvmqJNkrw7yQ1Jvprk2XNTriRpWLMZgWsd8IdVdXWSnYGrkny6qq4faPNrwNP6x3OAv+z/lSRtYWY846+qW6vq6v75fcBqYM9JzY4APlCdLwK7Jtl95NVKkoa2UWPuJlkEHABcMWnWnsBNA69v7qfdOmn5Y4BjABYuXMjExMTGVbsVWrNmTRP7ubVbtmzZrNplxcxtVq1aNWQ1mmut/1zOOviT7AR8DDi+qu7dlI1V1TnAOQBLliypFgY7bn1Q561FVc3YxmO57Wj9WM7qUz1J5tOF/oer6uNTNLkF2Hvg9V79NEnSFmY2n+oJsBJYXVVnTtPsIuA1/ad7fhm4p6punaatJGmMZtPV8zzg1cDXklzTTzsFeDJAVZ0N/CNwKHAD8CPgd0ZeqSRpJGYM/qq6HMgMbQr4g1EVJUmaO35zV5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGpPZ3JVwTjac3A58dywb37x2A+4YdxEaCY/ltqOVY7lPVT1x8sSxBX8rklxZVUvGXYeG57HcdrR+LO3qkaTGGPyS1BiDf+6dM+4CNDIey21H08fSPn5Jaoxn/JLUGINfkhpj8M+RJC9O8q9JbkiyfNz1aNMl+asktyX5+rhr0XCS7J1kVZLrk1yX5E3jrmkc7OOfA0nmAd8EfgW4GfgycGRVXT/WwrRJkjwfWAN8oKr2H3c92nRJdgd2r6qrk+wMXAX8Rms/m57xz41fAm6oqm9X1X8AHwGOGHNN2kRV9TngrnHXoeFV1a1VdXX//D5gNbDneKva/Az+ubEncNPA65tp8D+XtCVLsgg4ALhizKVsdga/pOYk2Qn4GHB8Vd077no2N4N/btwC7D3weq9+mqQxSzKfLvQ/XFUfH3c942Dwz40vA09L8pQkjwJeAVw05pqk5iUJsBJYXVVnjruecTH450BVrQPeAHyS7uLRhVV13Xir0qZKcj7wBeDpSW5OctS4a9Imex7wauDgJNf0j0PHXdTm5sc5JakxnvFLUmMMfklqjMEvSY0x+CWpMQa/JDXG4JcGJDktyYnjrkOaSwa/JDXG4FfTkrwmyVeTXJvkg5PmHZ3ky/28jyV5dD/95Um+3k//XD/tGUm+1H8h6KtJnjaO/ZFmwy9wqVlJngF8AvivVXVHkscDbwTWVNWfJXlCVd3Zt/3fwA+q6qwkXwNeXFW3JNm1qu5Ochbwxar6cH+bjnlV9cC49k3aEM/41bKDgb+tqjsAqmryPff3T/L5PuhfCTyjn/4vwPuTHA3M66d9ATglyUnAPoa+tmQGvzS99wNvqKpnAm8HFgBU1bHAW+juwHpV/5fB3wCHAw8A/5jk4PGULM3M4FfLPgu8PMkTAPqunkE7A7f2t/F95fqJSX62qq6oqrcCtwN7J3kq8O2qejfw98DPb5Y9kDbB9uMuQBqXqrouyenAZUkeAr4C3DjQ5H/Rjc50e//vzv30d/YXbwN8BrgWOAl4dZIfA/8O/Olm2QlpE3hxV5IaY1ePJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mN+f+TOatWvolawgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAGTCAYAAADX4IJXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAc80lEQVR4nO3dfZQddZ3n8feXJBAkEQQ0ijzER7YxiErr6MpqBxRdfGBmdhzNIupMlkzGMeIwzhCJM+Jou2TddVbRsxEmCooGFGXWAWVU6JZhFtAEQQOtc5TnB3kQgTRCIPG7f1Q15xK6c2/3vTe38+v365w6dFfVrfrWrfTnFr+q+/tFZiJJKsMuvS5AktQ5hrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdXVcRGREPL/XdfRSRAxExG3bWd719ygiboqI13VzH5p+DPWC1X/UD0fEaET8JiIuiogDel3XmIh4T0Rc3us6pJIY6uV7S2bOA54F3AWc3uN6uiYiZve6BqnXDPUZIjMfAc4HDhmbFxF7RsSXIuKeiLg5Ij4cEbtExN4RcVtEvKVeb15E/CIi3lX/flZErImI70XEpoj4QUQcNN5+t7OPPmAN8Kr6/yTun+D1z4mIy+r9fD8iPhcR59TLFtbNGEsj4hbg0nrbH673dXe97z3r9Z/UJNLYRBERp0bE+RFxXr2/qyPisIZ194uIb9THcmNEvL9h2e71+/KbiLgeeHkLp+WYiLghIu6NiE/Wte8aEfdFxKEN235GRPw2Ip4+wXt0QkSM1DVfHxEvG2edV0TEFRFxf0TcGRGfjYhd62UREf9Qv18PRsRPI2JRveyYepubIuL2iPhgC8elHjLUZ4iIeArwduDKhtmnA3sCzwVeC7wL+JPMvA/4U+DMiHgG8A/ANZn5pYbXHgd8DNgXuAb4ygS7nmgfI8By4IrMnJeZe03w+q8CPwT2AU4Fjh9nndcCfcAbgPfU0+J6n/OAz06w7fEcC3wd2Lve9z9FxJyI2AX4Z+Ba4NnAUcAHIuIN9es+Ajyvnt4AvLuFff0B0A+8rN7vn2bmo8C5wDsb1lsCXJKZ92y7gYh4G9X78i7gqcBbgV+Ps6+twF9Sna9X1fW/t152NPAa4IVU5+qPG7axFvizzJwPLAIubeG41EuZ6VToBNwEjAL3A48BdwCH1stmAY8ChzSs/2fAcMPvpwM/BW4H9mmYfxZwbsPv86hC44D69wSe32wfVOF7+XbqPxDYAjylYd45wDn1zwvrfT23YfklwHsbfj+4PvbZwABw2zjv0evqn08FrmxYtgtwJ/CfgN8DbtnmtR8Cvlj/fAPwxoZly7bd1zavzW3Wfy9VcDO2LyDq39cDfzzBdv4FOHE75/91Eyz7AHBB/fORwL8DrwR22Wa9W+pz9tRe/3t2am3ySr18v5/VVfBc4H3ADyLimVRXbHOAmxvWvZnqKnTMGVRXZ2dl5rZXf7eO/ZCZo8B9wH7brNPKPrZnP+C+zPztePudYN5+4+xvNrCgxX02HtfvgNvqbR4E7Fc3X9xfNxed0rDd/bapo7GGpvuq19+v3u9VwG+BgYj4D1QfkN+aYBsHAL9stqOIeGFEXBgRv4qIB4FPUJ0fMvNSqv+b+Rxwd0ScERFPrV/6X4BjgJvrZrZXtXBc6iFDfYbIzK2Z+U2qK+ojgHuprmAb28IPpLoqJyJmUYX6l4D3xpMfv3v8KZqImEfVXHHHNutsdx9UV6vbcyewd9109KT9Nh5ew893jLO/LVQ3iR8CHt9WfYzbtlM3HtcuwP71Nm8FbszMvRqm+Zl5TEOtjbUd2OTYtj2WA3ni+3c2VRPM8cD5Wd0TGc+tVE0+zfwf4GfACzLzqVQfSDG2MDM/k5mHU91zeSHw1/X8H2XmscAzgH8CvtbCvtRDhvoMUd8MOxZ4GjCSmVup/kAHI2J+faPzJKrmDaj+6JOqbf2TwJfqEBxzTEQcUd9s+xhVs8UTrqJb2MddwP5jN+y2lZk3UzU9nFrfQHwV8JYmh7oO+Mv6Bus8qivS8zJzC1UTw9yIeFNEzAE+DOy2zesPj4g/jOpJmg8Am6nuQ/wQ2BQRJ9c3RWdFxKKIGLsh+jXgQxHxtIjYH1jRpE6Av67XPwA4ETivYdk5VG3u76T6YJ3IPwIfjIjD63P8/Bj/pvV84EFgtL76//OxBRHx8oj4vfo9eQh4BPhd/Z4fFxF7ZuZj9et/18JxqZd63f7j1L2Jqk31Yap29U3ARuC4huVPowqPe6iu+P6O6oP+cOA3wPPr9WYB/wasqn8/i+rJle/V274MeE7DdrPhtePuo162K3ARVdPNvRMcw/OAf63rv4Tq/x7W1ssW1vua3bD+LvU+bq33eQ7wtIbl76G6qr4b+CBPblM/nypcNwE/Bl7W8Nr9qD40flW/P1c2vPYpVOF7P3A91ZVuszb191O1xf8a+F/ArG3W+X5dXzQ5z8uBn9fnYiPw0obzP1bfa6iu1Efr9/Pvqe9nUN00/Um97F6qm97z6vNzcX2sDwI/Ao7o9b9rp+1PYzdipJZFxFlUgfXhHuz7POBnmfmRLmz7VKoPo3c2W3dHiIgvAHf04n3Wzssva2haq5s37gNupHr07ljgtJ4WtQNExELgD4GX9rgU7WRsU9d090xgmKpp4DPAn2fmj3taUZdFxMeomlE+mZk39roe7VxsfpGkgnilLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdxar7N/94k3W2O+xcN0XVf/s5zdeUWmeoa6cRO/GYm7388NDMYqhLUkEMde1w9RX3h+ph0n4TEV+MiLn1sjdHxDV1n+X/LyJeXM//MlX3tP8c1fB3f1PP/3rdR/gDUQ1796I2a9vekHWnRsTXohoib1NEXBcR/Q3LXxYRP66XfT2qYfE+HhF7AN+h6o99tJ7G+p7fdaLtSVNhqKtXjqMa9u15VP13fzgiXgp8gWqknX2AzwPfiojdMvN4qlF43pLV8Hf/o97Od4AXUPX3fTUTD6vXVDQfsg6q4eLOBfaiGrjis/VrdwUuoOrBcm+q3hz/ACAzHwL+M1XnXPPq6Y7tbU+aKkNdvfLZzLw1q/FQB6nG4VwGfD4zr8pqUI+zqfozf+VEG8nML2TmpszcTNV17mFRDzQ9BS8Hnp6Zf5+Zj2bmDcCZwDsa1rk8M7+dVV/xXwbGBqZ+JVUHeZ/JzMeyGpDkhy3sc6LtSVNiL43qlfGGcjsIeHdENA4wsStPHiYPeHzkokHgbVQjGI0N4LAv8MAUanp8yLqGebOo+h8f86uGn39LNejG7LrG2/OJnSmNN/TetsbdXlaDekiTZqirV8Ybyu1WYDAzByd4zba9z/1Xqq54X0c1IMSeVAM6BFMzNmTdC6bw2juBZ0dENAR74/ih9pynHcLmF/XKX0TE/hGxN7CKarShM4Hl9dBqERF71EPPza9fcxfw3IZtzKdqnvk11chDn2izpmZD1m3PFVTjv74vImZHNXTgKxqW3wXs00bTkNQSQ1298lXgu1TDuf0S+HhmrgdOoLpZ+BvgF1TDz43571Q3VO+PiA9SDR93M9VA1tdTDS83ZXW79puBl1ANynEv1RigTYM4Mx+lGtRiKdWQdu8ELqT60CEzf0Z18/SGuv5xm5Skdtmfuna4iLgJ+G+Z+f1e19JNEXEVsCYzv9jrWjRzeKUudUhEvDYinlk3v7wbeDHVwM3SDmOoq3gRcUrDl34ap+90eFcHUz3jfj/wV8AfZeadHd6HtF02v0hSQbxSl6SCdOU59X333TcXLlzYjU1PGw899BB77LFHr8tQB3guyzITzueGDRvuzcynj7esK6G+cOFC1q9f341NTxvDw8MMDAz0ugx1gOeyLDPhfEbEzRMts/lFkgpiqEtSQQx1SSqIoS5JBWka6hFxcD0Szdj0YER8YAfUJkmapKZPv2Tmz6k6OBrrv/p2qhFeJEnTzGSbX44CfpmZEz5OI0nqncmG+juoug+VJE1DLff9Ug+sewfwosy8a5zly6jGmGTBggWHn3vuuZ2sc9oZHR1l3rx5vS5DHeC5LMtMOJ+LFy/ekJn94y2bTKgfC/xFZh7dbN3+/v70G6XaWXguyzITzmdETBjqk2l+WYJNL5I0rbUU6hGxB/B64JvdLUeS1I6WOvTKzIeAfbpciySpTV3ppVGSuiEiOrKdkgcHspsASTuNzGw6HXTyhU3XKZmhLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkFaCvWI2Csizo+In0XESES8qtuFSZImb3aL630auDgz/ygidgWe0sWaJElT1DTUI2JP4DXAewAy81Hg0e6WJUmailau1J8D3AN8MSIOAzYAJ2bmQ40rRcQyYBnAggULGB4e7nCp08vo6GjxxzhTeC7LM5PPZ2Tm9leI6AeuBF6dmVdFxKeBBzPzbyd6TX9/f65fv76zlU4zw8PDDAwM9LoMdYDnsiwLV17ETae9qddldFVEbMjM/vGWtXKj9Dbgtsy8qv79fOBlnSpOktQ5TUM9M38F3BoRB9ezjgKu72pVkqQpafXplxXAV+onX24A/qR7JUmSpqqlUM/Ma4Bx228kSdOH3yiVpIK02vwiSV112Ee/ywMPP9aRbS1ceVFbr99z9zlc+5GjO1LLjmaoS5oWHnj4sY48itiJR1Tb/VDoJZtfJKkghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JB7HpX0rQwv28lh569sjMbO7vdWgDa7wa4Fwx1SdPCppHT7E+9A2x+kaSCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQRx4WsWLiI5sJzM7sh1NrGMDPl/c3nb23H1OZ+roAUNdxWsWxgtXXtSRUezVnk6dg5l+PlsK9Yi4CdgEbAW2ZGZ/N4uSJE3NZK7UF2fmvV2rRJLUNm+USlJBWr1ST+C7EZHA5zPzjG1XiIhlwDKABQsWMDw83LEip6PR0dHij3Em8VyWZSafz1ZD/YjMvD0ingF8LyJ+lpmXNa5QB/0ZAP39/TkwMNDZSqeZ4eFhSj/GGePiizyXJZnh57Ol5pfMvL3+793ABcArulmUJGlqmoZ6ROwREfPHfgaOBjZ2uzBJ0uS10vyyALig/gLHbOCrmXlxV6uSJE1J01DPzBuAw3ZALZKkNvlIoyQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqSKvD2UnT0mEf/S4PPPxY29tZuPKitrex5+5zuPYjR7e9Hakdhrp2ag88/Bg3nfamtrbRqfFmO/HBILXL5hdJKoihLkkFMdQlqSC2qUvaaUREa+ut3v7yzOxANdOTV+qSdhqZ2XQaGhpquk7JDHVJKoihLkkFMdQlqSCGuiQVxFCXpIIY6pJUEJ9T105tft9KDj17ZfsbOrsTtQC01w+N1C5DXTu1TSOn2aGX1MDmF0kqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFMdQlqSCGuiQVpOVQj4hZEfHjiLiwmwVJkqZuMlfqJwIj3SpEktS+lkI9Ivan6ij6H7tbjiSpHa32p/6/gb8B5k+0QkQsA5YBLFiwgOHh4XZrm9ZGR0eLP8adRUf6Mb+4/W3sMQf/TUwDM/1vs2moR8Sbgbszc0NEDEy0XmaeAZwB0N/fn50YdGA669TACmrPTQPtb2PhyovaHmhD08dM/9tspfnl1cBbI+Im4FzgyIg4p6tVSZKmpGmoZ+aHMnP/zFwIvAO4NDPf2fXKJEmT5nPqklSQSQ08nZnDwHBXKpEktc0rdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekgszudQFSt0VE83VWN99OZnagGqm7vFJX8TJzu9PQ0FDTdQx07SwMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFaRrqETE3In4YEddGxHUR8dEdUZjUbevWrWPRokUcddRRLFq0iHXr1vW6JKltrXTotRk4MjNHI2IOcHlEfCczr+xybVLXrFu3jlWrVrF27Vq2bt3KrFmzWLp0KQBLlizpcXXS1DW9Us/KaP3rnHqydyPt1AYHB1m7di2LFy9m9uzZLF68mLVr1zI4ONjr0qS2tNT1bkTMAjYAzwc+l5lXjbPOMmAZwIIFCxgeHu5gmdPP6Oho8cdYspGREbZu3crw8PDj53Lr1q2MjIx4XndyM/1vs6VQz8ytwEsiYi/ggohYlJkbt1nnDOAMgP7+/hwYGOhwqdPL8PAwpR9jyfr6+pg1axYDAwOPn8uhoSH6+vo8rzu5mf63OamnXzLzfmAIeGNXqpF2kFWrVrF06VKGhobYsmULQ0NDLF26lFWrVvW6NKktTa/UI+LpwGOZeX9E7A68HmhhnBhp+hq7GbpixQpGRkbo6+tjcHDQm6Ta6bXS/PIs4Oy6XX0X4GuZeWF3y5K6b8mSJSxZsmTG/++6ytI01DPzJ8BLd0AtkqQ2+Y1SSSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKoihLkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakghrokFcRQl6SCGOqSVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXEUJekghjqklQQQ12SCmKoS1JBDHVJKkjTUI+IAyJiKCKuj4jrIuLEHVGYJGnyZrewzhbgrzLz6oiYD2yIiO9l5vVdrk2SNElNr9Qz887MvLr+eRMwAjy724VJkiYvMrP1lSMWApcBizLzwW2WLQOWASxYsODwc889t4NlTj+jo6PMmzev12WoAzyXZZkJ53Px4sUbMrN/vGUth3pEzAN+AAxm5je3t25/f3+uX79+0oXuTIaHhxkYGOh1GeoAz2VZZsL5jIgJQ72lp18iYg7wDeArzQK9dCtWrGDu3LksXryYuXPnsmLFil6XJEmPa3qjNCICWAuMZOanul/S9LVixQrWrFnD6tWrOeSQQ7j++us5+eSTATj99NN7XJ0ktXal/mrgeODIiLimno7pcl3T0plnnsnq1as56aSTmDt3LieddBKrV6/mzDPP7HVpkgS09vTL5ZkZmfnizHxJPX17RxQ33WzevJnly5c/Yd7y5cvZvHlzjyqSpCfyG6WTsNtuu7FmzZonzFuzZg277bZbjyqSpCdq5ctHqp1wwgmPt6EfcsghfOpTn+Lkk09+0tW7JPWKoT4JYzdDTznlFDZv3sxuu+3G8uXLvUkqadqw+WWSTj/9dB555BGGhoZ45JFHDHRJ04qhLkkFMdQlqSCGuiQVxFCXpIIY6pJUEB9pnEDV5U37JtO1sSS1yyv1CWTmdqeDTr6w6ToGuqQdzVCXpIIY6pJUEENdkgpiqEtSQWbk0y+HffS7PPDwY21vZ+HKi9p6/Z67z+Hajxzddh2SNGZGhvoDDz/GTae9qa1tdGJw23Y/FCRpWza/SFJBDHVJKsiMbH6Z37eSQ89e2f6Gzm63DoD2moEkqdGMDPVNI6fZpi6pSDa/SFJBDHVJKoihLkkFmZFt6tCh9uyL2//ykSR10owM9XZvkkL1odCJ7UhSJ9n8IkkFMdQlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSrIjPzyUSsiovk6q5tvJzM7UI0ktcYr9Qlk5nanoaGhpusY6JJ2NEN9ktatW8eiRYs46qijWLRoEevWret1SZL0OJtfJmHdunWsWrWKtWvXsnXrVmbNmsXSpUsBWLJkSY+rkySv1CdlcHCQtWvXsnjxYmbPns3ixYtZu3Ytg4ODvS5NkgBDfVJGRkY44ogjnjDviCOOYGRkpEcVSdITGeqT0NfXx+WXX/6EeZdffjl9fX09qkiSnqhpm3pEfAF4M3B3Zi7qfknT16pVq3j729/OHnvswS233MKBBx7IQw89xKc//elelyZJQGtX6mcBb+xyHTsdH1eUNB01DfXMvAy4bwfUMu0NDg5y3nnnceONN3LppZdy4403ct5553mjVNK00bFHGiNiGbAMYMGCBQwPD3dq09PGyMgIW7duZXh4mNHRUYaHh9m6dSsjIyNFHu9MMXYuVYaZfj47FuqZeQZwBkB/f38ODAx0atPTRl9fH7NmzWJgYIDh4WEGBgYYGhqir6+PEo93phg7lyrDTD+fPv0yCatWrWLp0qUMDQ2xZcsWhoaGWLp0KatWrep1aZIE+I3SSRn71uiKFSsYGRmhr6+PwcFBv00qadpoeqUeEeuAK4CDI+K2iFja/bKmryVLlrBx40YuueQSNm7caKBLmlaaXqlnpqklSTsJ29QlqSCGuiQVxFCXpIIY6pJUEENdkgpiqEtSQQx1SSqIoS5JBTHUJakg0Y3BHiLiHuDmjm94etkXuLfXRagjPJdlmQnn86DMfPp4C7oS6jNBRKzPzP5e16H2eS7LMtPPp80vklQQQ12SCmKoT90ZvS5AHeO5LMuMPp+2qUtSQbxSl6SCGOqSVBBDfQoi4o0R8fOI+EVErOx1PZqaiPhCRNwdERt7XYvaExEHRMRQRFwfEddFxIm9rqlXbFOfpIiYBfw78HrgNuBHwJLMvL6nhWnSIuI1wCjwpcxc1Ot6NHUR8SzgWZl5dUTMBzYAvz8T/y69Up+8VwC/yMwbMvNR4Fzg2B7XpCnIzMuA+3pdh9qXmXdm5tX1z5uAEeDZva2qNwz1yXs2cGvD77cxQ//xSNNRRCwEXgpc1eNSesJQl1SMiJgHfAP4QGY+2Ot6esFQn7zbgQMaft+/niephyJiDlWgfyUzv9nrenrFUJ+8HwEviIjnRMSuwDuAb/W4JmlGi4gA1gIjmfmpXtfTS4b6JGXmFuB9wL9Q3Yz5WmZe19uqNBURsQ64Ajg4Im6LiKW9rklT9mrgeODIiLimno7pdVG94CONklQQr9QlqSCGuiQVxFCXpIIY6pJUEENdkgpiqGtGiIhTI+KDva5D6jZDXZIKYqirSBHxroj4SURcGxFf3mbZCRHxo3rZNyLiKfX8t0XExnr+ZfW8F0XED+svs/wkIl7Qi+ORWuWXj1SciHgRcAHwHzPz3ojYG3g/MJqZ/zMi9snMX9frfhy4KzNPj4ifAm/MzNsjYq/MvD8iTgeuzMyv1N1CzMrMh3t1bFIzXqmrREcCX8/MewEyc9s+0xdFxL/WIX4c8KJ6/r8BZ0XECcCset4VwCkRcTJwkIGu6c5Q10x0FvC+zDwU+CgwFyAzlwMfpuqFc0N9Rf9V4K3Aw8C3I+LI3pQstcZQV4kuBd4WEfsA1M0vjeYDd9ZdtR43NjMinpeZV2Xm3wH3AAdExHOBGzLzM8D/BV68Q45AmqLZvS5A6rTMvC4iBoEfRMRW4MfATQ2r/C3VqDj31P+dX8//ZH0jNIBLgGuBk4HjI+Ix4FfAJ3bIQUhT5I1SSSqIzS+SVBBDXZIKYqhLUkEMdUkqiKEuSQUx1CWpIIa6JBXk/wNTvDlGM3MthAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAGTCAYAAAAvAHmgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcRElEQVR4nO3de7RkZX3m8e9D00YFxAtJK9DSXjDTinHUjpeRMQcxxkCUTBIzdghqQux410zM2IrjJUknZJxJZimuGJImQDStiRqHpI2XaBdIosglIJdOslAhgHgB5HKAUSC/+WPvJsXhdJ+CqtN1Tr/fz1q1qNr7rb1/++zmOfu8u+p9U1VIktqx17QLkCTtXga/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5NRZJK8vhp1zFNSWaSXL2L9Yv+M0pyRZLnL+Y+tPQY/I3r/8e/Pclsku8m2Zpk9bTr2iHJK5KcPe06pD2JwS+AF1XVvsCjgG8B75tyPYsmyd7TrkGaNoNfd6uq/wd8FHjijmVJ9k9yepLvJLkyyduT7JXk4UmuTvKivt2+SS5P8rL+9alJPpDks0luSXJmkkPm2+8u9rEW+ADw7P4vkht38v7HJDmr38/fJXl/kg/269b0XSbHJ/lX4PP9tt/e7+vb/b7379vfq/tluDskybuSfDTJR/r9XZDkKUNtD0zysf5Yvp7kDUPrHtT/XL6b5DLgR0c4LUcl+VqS65K8p6/9AUluSPLkoW3/UJLbkvzgTn5Gr0yyva/5siRPm6fNM5J8McmNSa5NclKSB/TrkuQP+p/XzUkuTnJYv+6ofpu3JLkmyZtHOC5NkcGvuyV5MPBfgS8NLX4fsD/wWODHgJcBv1RVNwC/DPxxkh8C/gC4sKpOH3rvscBvAQcAFwIf2smud7aP7cCrgC9W1b5V9dCdvP/PgS8DjwDeBRw3T5sfA9YCPwG8on8c0e9zX+CknWx7PscAfwk8vN/3J5KsTLIX8NfARcBBwJHAm5L8RP++dwKP6x8/Abx8hH39F2Ad8LR+v79cVd8HPgz84lC79cDnquo7czeQ5CV0P5eXAQ8BXgxcP8++7gJ+je58Pbuv/zX9uhcAzwWeQHeufn5oG5uBX62q/YDDgM+PcFyapqry0fADuAKYBW4E7gC+ATy5X7cC+D7wxKH2vwoMhl6/D7gYuAZ4xNDyU4EPD73ely5YVvevC3j8QvugC+izd1H/o4E7gQcPLfsg8MH++Zp+X48dWv854DVDr3+4P/a9gRng6nl+Rs/vn78L+NLQur2Aa4H/DDwT+Nc5730r8Kf9868BLxxat2Huvua8t+a0fw1duLNjX0D61+cBP7+T7XwaeOMuzv/zd7LuTcBf9c+fB/wL8Cxgrznt/rU/Zw+Z9r9nH6M9vOIXwE9XdzX9QOB1wJlJHkl35bcSuHKo7ZV0V7M7nEx3lXdqVc29irxqx5OqmgVuAA6c02aUfezKgcANVXXbfPvdybID59nf3sCqEfc5fFz/Blzdb/MQ4MC+q+TGvmvqbUPbPXBOHcM1LLivvv2B/X7PAW4DZpL8B7pfomfsZBurga8utKMkT0jyN0m+meRm4Hfozg9V9Xm6v4reD3w7yclJHtK/9WeBo4Ar+y69Z49wXJoig193q6q7qurjdFfmhwPX0V0JD/fNP5ru6p4kK+iC/3TgNbn3Rw/v/nRQkn3puka+MafNLvdBd9W7K9cCD++7qe613+HDG3r+jXn2dyfdje1bgbu31R/j3H7z4ePaCzi43+ZVwNer6qFDj/2q6qihWodre/QCxzb3WB7NPX9+p9F19xwHfLS6ezTzuYque2khfwj8E3BoVT2E7pdWdqysqvdW1dPp7gE9AfiNfvm5VXUM8EPAJ4C/GGFfmiKDX3frb+AdAzwM2F5Vd9H9T7wpyX79zdn/RteVAl0wFF1f/3uA0/ug3OGoJIf3Nwh/i66L5B5X4yPs41vAwTtuMs5VVVfSdXO8q7/p+WzgRQsc6hbg1/qbwvvSXdl+pKrupOvOeGCSo5OsBN4O/MCc9z89yc+k+4TQm4Dv0d0X+TJwS5K39DdyVyQ5LMmOm7h/Abw1ycOSHAy8foE6AX6jb78aeCPwkaF1H6S7B/CLdL98d+ZPgDcneXp/jh+f+W+07wfcDMz2f0W8eseKJD+a5Jn9z+RW4P8B/9b/zI9Nsn9V3dG//99GOC5N07T7mnxM90HXx3s7XT//LcAlwLFD6x9GFzDfobtyfAfdBcPTge8Cj+/brQD+Hjihf30q3SdyPttv+yzgMUPbraH3zruPft0DgK103UTX7eQYHgd8oa//c3R/hWzu163p97X3UPu9+n1c1e/zg8DDhta/gu7q/NvAm7l3H/9H6QL4FuAfgacNvfdAul8s3+x/Pl8aeu+D6QL6RuAyuivmhfr430B3b+B64H8DK+a0+bu+vixwnl8F/HN/Li4Bnjp0/nfU91y6K/7Z/uf5m/T3V+hu9H6lX3cd3Y36ffvz86n+WG8GzgUOn/a/ax+7fuy4MSRNVJJT6ULt7VPY90eAf6qqdy7Ctt9F9wvrFxdquzskOQX4xjR+zlq+/DKLlr2+K+UG4Ot0Hzs8BjhxqkXtBknWAD8DPHXKpWiZsY9fe4JHAgO6boj3Aq+uqn+cakWLLMlv0XXZvKeqvj7terS82NUjSY3xil+SGmPwS1JjDH5JaozBL0mNMfglqTEGv9Trx8r/7Qlt69gkn9nF+kGSX9kdtUhzGfzaY2WK88lW1Yeq6gWjtI3TS2o3M/glqTEGv5a8/sr9rf30ft9N8qdJHtiv+6kkF/bj3/9Dkh/pl/8Z3TDGf51u2sb/3i//y368+ZvSTdf4pPtYy5lJfrZ//px00zoe3b8+MsmF/fN7XMUn+fEk/9Tv9yT64Y6z6+klH5Zka7opDc9JMsrQytKCDH4tF8fSTVf4OLqx4N+e5KnAKXSzPz0C+CPgjCQ/UFXH0c0M9aLqpm38n/12/hY4lG7s+AvY+XSQO3Mm3Sxd0E3n+DW6US13vD5z7huSHAB8nG6I5wPoJkV5DkDtenrJlwLvphu99HJg032sVZqXwa/l4qSquqq6uX430c0xuwH4o6o6p7pJZE6jGxv/WTvbSFWdUlW3VNX36IZYfkr6idZHdCZdwEMX+L879Hre4KebnerSqvpodWPW/x+6YZsX8ldV9eXq5gn4EPAf70Od0k4Z/Fou5puC8BDg13PPqQ5Xc+/pHYFuNq0kJyb5arqpBa/oVx1wH+r4IvCEJKvogvh0YHV/Vf8MunkH5rrHlIvVDZA13/SQcw3/criNbvx7aWwOy6zlYr4pCK8CNlXVzrpA5o5A+At0QzY/ny7096ebQCSMqKpuS3I+3WxYl1TV95P8A92sYV+tquvmeds9plxMkjnH40iJ2q284tdy8dokByd5OHAC3QxYfwy8qp8SMEn26adM3K9/z7eAxw5tYz+6rqDr6WbD+p37WcuZ9JPS968Hc17PtRV40tB0jW+gG0p6h11OLylNmsGv5eLPgc/Q3Uz9KvDbVXUe8ErgJLor98vppk3c4XfpbgLfmOTNdN0yV9JN5H4Z3bSI98eZdL9EztrJ63vo/wp4Cd3kMNfT3Vz++6EmnwcuBb6ZZL6/GKSJcjx+LXlJrgB+par+btq1SHsCr/glqTEGvzRHkrf1X6aa+/jbadcmTYJdPZLUGK/4JakxU/sc/wEHHFBr1qyZ1u53m1tvvZV99tln2mVoAjyXe45WzuX5559/XVX94NzlUwv+NWvWcN55501r97vNYDBgZmZm2mVoAjyXe45WzmWSK+dbblePJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1ZsHgT7I6ybYklyW5NMkb52kzk+SmJBf2j3csTrmSpHGNMmTDncCvV9UF/ZR25yf5bFVdNqfdF6rqpyZfoiRpkha84q+qa6vqgv75LcB24KDFLkyStDjuUx9/kjXAU4Fz5ln97CQXJfnbJE+aRHGSpMkbeXTOJPsCHwPeVFU3z1l9AXBIVc0mOQr4BN2E0nO3sQHYALBq1SoGg8H9LHv5mJ2dbeI4W+C5XD6OOOKIiWxn27ZtE9nOUjPSDFxJVgJ/A3y6qn5/hPZXAOuq6rqdtVm3bl05LLOWE8/lnmPNxq1cceLR0y5j0SU5v6rWzV0+yqd6AmwGtu8s9JM8sm9Hkmf0271+vJIlSYthlK6e5wDHARcnubBf9jbg0QBV9QHg54BXJ7kTuB14aTmZryQtSQsGf1WdDWSBNicBJ02qKEnS4vGbu5LUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjRplsXZKWjKe8+zPcdPsdY29nzcatY71//wet5KJ3vmDsOqbB4Je0rNx0+x1cceLRY21jMBgwMzMz1jbG/cUxTXb1SFJjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhqzYPAnWZ1kW5LLklya5I3ztEmS9ya5PMlXkjxtccqVJI1r7xHa3An8elVdkGQ/4Pwkn62qy4ba/CRwaP94JvCH/X8lSUvMglf8VXVtVV3QP78F2A4cNKfZMcDp1fkS8NAkj5p4tZKksY1yxX+3JGuApwLnzFl1EHDV0Our+2XXznn/BmADwKpVqxgMBvet2mVodna2ieNsgedyadhv7UaefNrG8Td02rh1wGCwz/h1TMHIwZ9kX+BjwJuq6ub7s7OqOhk4GWDdunU1MzNzfzazrAwGA1o4zhZ4LpeGWzaeyBUnHj3WNiZxLtds3MrMy8fbxrSM9KmeJCvpQv9DVfXxeZpcA6ween1wv0yStMSM8qmeAJuB7VX1+ztpdgbwsv7TPc8Cbqqqa3fSVpI0RaN09TwHOA64OMmF/bK3AY8GqKoPAJ8EjgIuB24DfmnilUqSJmLB4K+qs4Es0KaA106qKEnS4vGbu5LUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGjPKnLvSHi/Z5eyi90k3E6kW05qNW8ffyKfG28b+D1o5fg1TYvBLjBbWazZu5YoTj94N1WhXJnEOWj+XdvVIUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGrNg8Cc5Jcm3k1yyk/UzSW5KcmH/eMfky5QkTcreI7Q5FTgJOH0Xbb5QVT81kYokSYtqwSv+qjoLuGE31CJJ2g1GueIfxbOTXAR8A3hzVV06X6MkG4ANAKtWrWIwGExo90vX7OxsE8e51L32c7dy6x3jb2fNxq1jvX+flfD+I/cZvxCNreX/LycR/BcAh1TVbJKjgE8Ah87XsKpOBk4GWLduXc3MzExg90vbYDCgheNc6m791FauOPHosbYxiXO5ZuNW/z0sBZ9q+zyM/ameqrq5qmb7558EViY5YOzKJEmLYuzgT/LIJOmfP6Pf5vXjbleStDgW7OpJsgWYAQ5IcjXwTmAlQFV9APg54NVJ7gRuB15aVbVoFUuSxrJg8FfV+gXWn0T3cU9J0jLgN3clqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWrMpAZpk6Qlox9MYNdtfm/h7eyp30X1il/SHqeqdvnYtm3bgm321NAHg1+SmmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYxyrR03Yb+1GnnzaxvE3dNq4dQAcPX4d0hgMfjXhlu0ncsWJ4wXuYDBgZmZmrG2s2bh1rPdLk2BXjyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGOGSDmjGR4RI+Nd429n/QyvFrkMZk8KsJ447TA90vjklsR5o2u3okqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktSYBYM/ySlJvp3kkp2sT5L3Jrk8yVeSPG3yZUqSJmWUK/5TgRfuYv1PAof2jw3AH45fliRpsSwY/FV1FnDDLpocA5xenS8BD03yqEkVKEmarL0nsI2DgKuGXl/dL7t2bsMkG+j+KmDVqlUMBoMJ7H5pm52dbeI4l7sjjjhipHb5vYXbbNu2bcxqtNha//9yEsE/sqo6GTgZYN26dTUzM7M7dz8Vg8GAFo5zuauqBdt4LvccrZ/LSXyq5xpg9dDrg/tlkqQlaBLBfwbwsv7TPc8Cbqqqe3XzSJKWhgW7epJsAWaAA5JcDbwTWAlQVR8APgkcBVwO3Ab80mIVK0ka34LBX1XrF1hfwGsnVpEkaVH5zV1JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMM/kWyZcsWDjvsMI488kgOO+wwtmzZMu2SJAmAvaddwJ5oy5YtnHDCCWzevJm77rqLFStWcPzxxwOwfv36KVcnqXVe8S+CTZs2sXnzZo444gj23ntvjjjiCDZv3symTZumXZokGfyLYfv27Rx++OH3WHb44Yezffv2KVUkSf/O4F8Ea9eu5eyzz77HsrPPPpu1a9dOqSJJ+ncG/yI44YQTOP7449m2bRt33nkn27Zt4/jjj+eEE06YdmmS5M3dxbDjBu7rX/96tm/fztq1a9m0aZM3diUtCQb/Ilm/fj3r169nMBgwMzMz7XIk6W529UhSYwx+SWqMwS9JjTH4JakxIwV/khcm+ecklyfZOM/6VyT5TpIL+8evTL7U5cWxeiQtVQt+qifJCuD9wI8DVwPnJjmjqi6b0/QjVfW6Rahx2XGsHklL2ShX/M8ALq+qr1XV94EPA8csblnLm2P1SFrKRvkc/0HAVUOvrwaeOU+7n03yXOBfgF+rqqvmNkiyAdgAsGrVKgaDwX0ueDnYvn07d911F4PBgNnZWQaDAXfddRfbt2/fY4+5BTvOpZa/1s/lpL7A9dfAlqr6XpJfBU4Dnje3UVWdDJwMsG7dutpTv9i0du1aVqxYwczMzN1f4Nq2bRtr1671y1zLmF/G23O0fi5H6eq5Blg99Prgftndqur6qvpe//JPgKdPprzlybF6JC1lo1zxnwscmuQxdIH/UuAXhhskeVRVXdu/fDHQ9PjDjtUjaSlbMPir6s4krwM+DawATqmqS5P8JnBeVZ0BvCHJi4E7gRuAVyxizcuCY/VIWqpG6uOvqk8Cn5yz7B1Dz98KvHWypUmSFoPf3JWkxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNcc7dMSSZyHaqaiLbkaRReMU/hqpa8HHIW/5mwTaStDsZ/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMcpG0XnvLuz3DT7XeMvZ01G7eO9f79H7SSi975grHrkCQw+Hfpptvv4IoTjx5rG4PBgJmZmbG2Me4vDkkaZlePJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMY4ZMMu7Ld2I08+beP4Gzpt3DoAxhs6QpJ2MPh34eKXXzz2NtZs3Dr2eD+SNEl29UhSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjEM2jCHJaO1+b9frq2oC1UjSaLziH0NVLfjYtm3bgm0kaXcy+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JasxIwZ/khUn+OcnlSe41+3iSH0jykX79OUnWTLxSSdJELBj8SVYA7wd+EngisD7JE+c0Ox74blU9HvgDYIHvqkqSpmWUK/5nAJdX1deq6vvAh4Fj5rQ5Bjitf/5R4MiMOp6BJGm3GmWsnoOAq4ZeXw08c2dtqurOJDcBjwCuG26UZAOwAWDVqlUMBoP7V/UyMjs728RxtsBzuedo/Vzu1kHaqupk4GSAdevW1czMzO7c/VQMBgNaOM4WeC73HK2fy1G6eq4BVg+9PrhfNm+bJHsD+wPXT6JASdJkjXLFfy5waJLH0AX8S4FfmNPmDODlwBeBnwM+XwsMO3n++edfl+TK+17ysnMAc7q8tGx5LvccrZzLQ+ZbuGDw9332rwM+DawATqmqS5P8JnBeVZ0BbAb+LMnlwA10vxwW2u4P3pfql6sk51XVumnXofF5LvccrZ/LOB784mr9H9iexHO552j9XPrNXUlqjMG/+E6edgGaGM/lnqPpc2lXjyQ1xit+SWqMwS9JjTH4F8lCI5pq+UhySpJvJ7lk2rVoPElWJ9mW5LIklyZ547Rrmgb7+BdBP6LpvwA/Tje20bnA+qq6bKqF6X5J8lxgFji9qg6bdj26/5I8CnhUVV2QZD/gfOCnW/t/0yv+xTHKiKZaJqrqLLovJmqZq6prq+qC/vktwHa6QSabYvAvjvlGNG3uH5e0lPUTRj0VOGfKpex2Br+k5iTZF/gY8Kaqunna9exuBv/iGGVEU0lTkGQlXeh/qKo+Pu16psHgXxx3j2ia5AF0g9adMeWapOb1MwNuBrZX1e9Pu55pMfgXQVXdCewY0XQ78BdVdel0q9L9lWQL3ZDjP5zk6iTHT7sm3W/PAY4Dnpfkwv5x1LSL2t38OKckNcYrfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj80pAk70ry5mnXIS0mg1+SGmPwq2lJXpbkK0kuSvJnc9a9Msm5/bqPJXlwv/wlSS7pl5/VL3tSki/3Xwj6SpJDp3E80ij8ApealeRJwF8B/6mqrkvycOANwGxV/a8kj6iq6/u2vw18q6rel+Ri4IVVdU2Sh1bVjUneB3ypqj7UD9Oxoqpun9axSbviFb9a9jzgL6vqOoCqmjvm/mFJvtAH/bHAk/rlfw+cmuSVwIp+2ReBtyV5C3CIoa+lzOCXdu5U4HVV9WTg3cADAarqVcDb6UZgPb//y+DPgRcDtwOfTPK86ZQsLczgV8s+D7wkySMA+q6eYfsB1/bD+B67Y2GSx1XVOVX1DuA7wOokjwW+VlXvBf4v8CO75Qik+2HvaRcgTUtVXZpkE3BmkruAfwSuGGryP+hmZ/pO/9/9+uXv6W/eBvgccBHwFuC4JHcA3wR+Z7cchHQ/eHNXkhpjV48kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY35/4CKviJcRfe8AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "for col in features:\n",
    "    data.boxplot(column=col, by=label, figsize=(6,6))\n",
    "    plt.title(col)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从结果图中可以看出这三种鸢尾\n",
    "- 在萼片长度上，山鸢尾要比变色鸢尾与弗吉尼亚鸢尾短，弗吉尼亚鸢尾整体萼片长度比变色鸢尾长，但二者萼片长度有重合部分\n",
    "- 在萼片宽度上，山鸢尾整体上要比变色鸢尾与弗吉尼亚鸢尾宽，弗吉尼亚鸢尾整体上萼片宽度大于变色鸢尾但二者有宽度重合部分\n",
    "- 在花瓣长度上，整体上呈现 `山鸢尾 < 变色鸢尾 < 弗吉尼亚鸢尾`\n",
    "- 在花瓣宽度上，整体上呈现 `山鸢尾 < 变色鸢尾 < 弗吉尼亚鸢尾`\n",
    "\n",
    "综上，可以预见的是在鸢尾花瓣的两个特征上对鸢尾进行区分的效果是值得期待的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 分配训练集与测试集\n",
    "- 总共 150 条鸢尾数据， 分配 `70%(105)`作训练集，`30%(45)` 作为测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training cases: 105\n",
      "Test cases: 45\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split data 70%-30% into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "print ('Training cases: %d\\nTest cases: %d' % (X_train.shape[0], X_test.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 标准化特征并训练模型 \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pipeline(steps=[('preprocessor',\n",
      "                 ColumnTransformer(transformers=[('preprocess',\n",
      "                                                  Pipeline(steps=[('scaler',\n",
      "                                                                   StandardScaler())]),\n",
      "                                                  [0, 1, 2])])),\n",
      "                ('regressor', LogisticRegression())])\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# 定义数字列的预处理（缩放它们）\n",
    "feature_columns = [0, 1, 2]\n",
    "feature_transformer = Pipeline(steps=[\n",
    "    ('scaler', StandardScaler())\n",
    "])\n",
    "\n",
    "# 创建预处理步骤\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('preprocess', feature_transformer, feature_columns)])\n",
    "\n",
    "# 创建训练管线\n",
    "pipeline = Pipeline(steps=[('preprocessor', preprocessor),\n",
    "                           ('regressor', LogisticRegression(solver='lbfgs', multi_class='auto'))])\n",
    "\n",
    "\n",
    "# 拟合管道以在训练集上训练线性回归模型\n",
    "model = pipeline.fit(X_train, y_train)\n",
    "print(model)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 评估模型\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall Accuracy: 0.9555555555555556\n",
      "Overall Precision: 0.9511784511784511\n",
      "Overall Recall: 0.9511784511784511\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVkAAAEzCAYAAACbnXDnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmqElEQVR4nO3dd5xkVZ3+8c8zQxaQMCMSRFglLLDEwRUDghGEFXRBBkEFWTGsiAq6RhTDT3FdhcUVdkijgEMGRUWSgBhhZhyUIRgWEASFAQUkh+f3xz0NNU13VXVNVdft6ufN676664ZT3zp0f+f0ueeeI9tERERvTOl3ABERgyxJNiKih5JkIyJ6KEk2IqKHkmQjInooSTYiooeSZGPcSFpe0vmS7pV05hKUs4+ki7oZWz9IukDS2/sdR/RWkmw8g6S3SJor6e+S7ijJ4GVdKHoPYA1gddt7dlqI7VNtv7YL8SxG0g6SLOncYfu3KPsvb7Ocz0g6pdV5tne2/c0Ow40JIkk2FiPpQ8CRwP+jSojrAt8AdutC8c8Hfmv78S6U1St3AdtJWr1h39uB33brDVTJ795kYTtbNmwDPBv4O7Bnk3OWpUrCt5ftSGDZcmwH4DbgEOBO4A5g/3LscOBR4LHyHgcAnwFOaSh7PcDAUuX1fsD/AfcDNwH7NOz/ScN1LwGuBu4tX1/ScOxy4HPAT0s5FwHTRvlsQ/EfC/x72TcV+BNwGHB5w7lHAbcC9wHzgJeX/TsN+5zXNMTxhRLHQ8ALy75/K8ePAc5uKP8I4FJA/f65yLZkW/41jUbbAcsB5zY55xPAi4EtgS2AFwGfbDj+XKpkvTZVIv0fSava/jRV6/h02yvaPqFZIJKeBfw3sLPtlagS6YIRzlsN+H45d3Xgq8D3h7VE3wLsDzwHWAY4tNl7A98C3la+fx1wLdU/KI2upqqD1YBvA2dKWs72D4d9zi0arnkrcCCwEnDLsPIOAf5J0n6SXk5Vd293ybgxcSXJRqPVgUVu/uf8PsBnbd9p+y6qFupbG44/Vo4/ZvsHVK25jTqM50lgM0nL277D9sIRztkF+J3tk20/bnsOcAPwLw3nnGT7t7YfAs6gSo6jsv0zYDVJG1El22+NcM4ptu8u7/lfVC38Vp9ztu2F5ZrHhpX3IFU9fhU4BTjI9m0tyosJIEk2Gt0NTJO0VJNz1mLxVtgtZd9TZQxL0g8CK441ENsPAHsB7wbukPR9SRu3Ec9QTGs3vP5zB/GcDLwP2JERWvaSDpV0fRkp8Teq1vu0FmXe2uyg7V9SdY+I6h+DGABJstHo58AjwO5Nzrmd6gbWkHV55p/S7XoAWKHh9XMbD9q+0PZrgDWpWqfHtRHPUEx/6jCmIScD7wV+UFqZTyl/zn8EeDOwqu1VqPqDNRT6KGU2/dNf0r9TtYhvL+XHAEiSjafYvpfqBs//SNpd0gqSlpa0s6Qvl9PmAJ+UNF3StHJ+y+FKo1gAbC9pXUnPBj42dEDSGpJ2K32zj1B1Ozw5Qhk/ADYsw86WkrQXsAnwvQ5jAsD2TcArqPqgh1sJeJxqJMJSkg4DVm44/hdgvbGMIJC0IfB5YF+qboOPSNqys+ijTpJkYzGlf/FDVDez7qL6E/d9wHnllM8Dc4FfA78B5pd9nbzXxcDppax5LJ4Yp5Q4bgfuoUp47xmhjLuBXaluHN1N1QLc1faiTmIaVvZPbI/USr8Q+CHVsK5bgIdZvCtg6EGLuyXNb/U+pXvmFOAI29fY/h3wceBkScsuyWeI/lNuXkZE9E5ashERPZQkGxHRQ0myERE9lCQbEdFDzQadR6Gln2Utt0q/w6itrTZcs98hxACYP3/eItvTl6SMqSs/3378oZbn+aG7LrS905K8V7uSZNug5VZh2W3e3e8wauunl3yq3yHEAFh+aQ1/cm/M/PjDLLvxzJbnPfyro1s9ndc1SbIRMTgESC1PG09JshExWGo2VW+SbEQMlrRkIyJ6RTBlar+DWEySbEQMDpHugoiI3lG6CyIieiot2YiIHqpZS7ZeKT8iYomoasm22topSTpR0p2Srh22/yBJN0ha2DCZ/ajSko2IwSG6ObpgNvB1GhbSlLQjsBuwhe1HJD2nVSFJshExQNS1PlnbP5a03rDd7wG+ZPuRcs6drcpJd0FEDJYpar1VqzLPbdgObLP0DYGXS/qlpCskbdvqgrRkI2JwtD9OdpHtGR28w1LAasCLgW2BMyT9g5us45WWbEQMFqn11rnbgHNcuYpqBeWmM3olyUbEAOne6IJRnAfsCE8t474M0HRl5HQXRMRg6dLoAklzgB2o+m9vAz4NnAicWIZ1PQq8vVlXASTJRsQgWfLugKfY3nuUQ/uOpZwk2YgYLHmsNiKih2r2WG2SbEQMkO49jNAtSbIRMTi6+1htVyTJRsQASUs2IqK30icbEdFDaclGRPRQWrIRET2i9MlGRPSUpiTJRkT0hACluyAiokdUthpJko2IAaK0ZCMieqluSbYnPcSSLpP0umH7PiDpmDGU8VlJr25xzn6S1uogvmmSHpP07rFeGxH1JqnlNp56dRtuDjBz2L6ZZX9LkqbaPsz2JS1O3Q8Yc5IF9gR+AYw2X2RETEQCTVHLbTz1KsmeBewiaRmAsqzuWsCVko4pq0MulHT40AWSbpZ0hKT5wJ6SZkvaoxzbpqwMOU/ShZLWLMdmAKdKWiBpF0nnNZT3GknnjhLf3sAhwNqS1unB54+IPhCtW7ED0ZK1fQ9wFbBz2TUTOKMs0/CJskrk5sArJG3ecOndtre2fdrQDklLA0cDe9jehmr5hy/YPguYC+xje0vgB8DGkqaXS/cv5y5G0vOANcsiaGcAe3Xrc0dE/3UryUo6UdKdZamZ4ccOkWRJTRdRhN4upNjYZdDYVfDm0lr9FbApsEnDNaePUM5GwGbAxZIWAJ8EntH6LAn8ZGBfSasA2wEXjFDeXlTJFeA0RukykHTg0JrsfuyBUT5iRNRNF1uys4GdRij/ecBrgT+2U0gvRxd8B/iapK2BFWzPk7Q+cCiwre2/SpoNLNdwzUjZTMBC29u18Z4nAecDDwNn2n58hHP2Bp4raZ/yei1JG9j+XeNJtmcBswCmrLR204XSIqI+utUdYPvHpatzuK8BH6HKcS31rCVr++/AZVR/sg+1YlemSqT3SlqDp7sTmrkRmC5pO6i6DyRtWo7dD6zU8J63A7dTtXZPGl5QWcJ3Rdtr217P9nrAF8kNsIjB0P6Nr2lDf6mW7cC2ipd2A/5k+5p2Q+r1ONk5wLmUbgPb10j6FXADcCvw01YF2H603OT6b0nPpor5SGAhVXP+WEkPAdvZfgg4FZhu+/oRitu7xNPobKpuis+O+dNFRK2o/YcRFpV7Q+2XLa0AfJyqq6BtPU2yts9j2ENutvcb5dz1RjvP9gJg+xGuOZsqSTZ6GXDcKO9x+Aj7fg3840jnR8TE08PRAy8A1geuKe+xDjBf0ots/3m0iwbqiS9J86i6Iw7pdywR0Sc9yrG2fwM856m3kW4GZthe1Oy6es0JtoRsb2N7e9uP9DuWiOgDdXUI1xzg58BGkm6TdEAnIQ1USzYiooujC5reEB/exTmaJNmIGBhCTMmk3RERPVSvSbiSZCNigKh+Ux0myUbEQEmSjYjooSTZiIgeGu/5YltJko2IgdGP+WJbSZKNiIGSJBsR0UNJshERvVSvHJskGxGDJS3ZiIgekWBKRhdERPRKRhdERPRUzXJskmxEDJa0ZCMiekX1a8nWa+LFiIglIKobX622tsqSTpR0p6RrG/b9p6QbJP1a0rmSVmlVTpJsRAyUbiVZqtWwdxq272JgM9ubA78FPtYynrEEHxFRa6W7oNXWDts/Bu4Ztu8i24+Xl7+gWrG2qfTJRsTAEG3f+JomaW7D61m2Z43x7d4BnN7qpCTZiBggbY+TXWR7RsfvIn0CeBw4tdW5SbIRMVB6PbpA0n7ArsCrbLvV+UmyETE4evxYraSdgI8Ar7D9YDvX5MZXRAyMoT7ZVltbZUlzgJ8DG0m6TdIBwNeBlYCLJS2QdGyrctKSjYiB0q3uAtt7j7D7hLGWkyQbEQMlj9VGRPRQzXJskmxEDBClJTshbbXhmvz0kk/1O4zaWnXb9/U7hNq79coj+x3CpCDG9NjsuEiSjYiBUrOGbJJsRAyWunUXtBwnK+lfJGU8bUTUXxcniOmWdpLnXsDvJH1Z0sa9DigiolPdfBihW1omWdv7AlsBfwBmS/q5pAMlrdTz6CIixmjCJVkA2/cBZwGnAWsCbwTmSzqoh7FFRIxZFyft7k48rU6QtJukc4HLgaWBF9neGdgCOKS34UVEjEEN+2TbGV3wJuBrZZbwp9h+sEyYEBFRC2p/Ptlx0053wZ+HJ1hJRwDYvrQnUUVEdKhuLdl2kuxrRti3c7cDiYjohilSy208jdpdIOk9wHuBF0j6dcOhlYCf9jqwiIixUo8n7e5Esz7ZbwMXAF8EPtqw/37b94x8SUREf9Usx47eXWD7Xts3l4lrnwe80vYtwBRJ649bhBERYzDhxslK+jTwH8DHyq5lgFN6GVRERKe6deNL0omS7pR0bcO+1SRdLOl35euqrcpp58bXG4E3AA8A2L6dql82IqJWRBnG1eK/Ns0Gdhq276PApbY3AC5l8a7UEbWTZB8ty94aQNKz2o0wImK8TVHrrR1l6Orw+0+7Ad8s338T2L1VOe08jHCGpP8FVpH0TuAdwHHthRkRMY7U9mOz0yTNbXg9y/asNq5bw/Yd5fs/A2u0uqBlkrX9FUmvAe4DNgIOs31xG8FERIwrQbvjYBfZnrEk72XbktzqvLYm7S5JNYk1Imqvx4MH/iJpTdt3SFoTuLPVBaP2yUr6Sfl6v6T7Grb7Jd3XxaAjIrqmx0O4vgu8vXz/duA7rS4YtSVr+2Xla0YSRMSE0M25CSTNAXag6r+9Dfg08CWq+1QHALcAb25VTtPuAklTgYW2syJCREwI3ZqboDyINZJXjaWcpkO4bD8B3Chp3bEUGhHRLxNmgpgGqwILJV1FeSABwPYbehZVREQHqtEF/Y5ice0k2U/1PIqIiG7ow9wErbQzTvaK8QgkIqIbapZj25og5sWSrpb0d0mPSnoiQ7gioq7qNgtXO90FXwdmAmcCM4C3ARv2MqiIiE4ImFqzTtl2lwT/PTDV9hO2T+KZM9NERNSC2tjGUzst2QclLQMskPRl4A7aTM4REeNJ6t442W5p9ljttuXbt5bz3kc1hOt5wL/2PrSIiLGr22q1zVqysyStCJwGzLF9HXD4+IQVEdGZug3harbG11bArsDjwFmSrpH0UUnrtSpU0mWSXjds3wckHdNuYJI+K+nVLc7ZT9Ja7ZZZrrlc0o2SFki6XtKBY7k+Iuqtbi3ZVo/V3mj7cNubUI0qeDZwqaRWS4LPoRqR0Ghm2d+SpKm2D7N9SYtT9wPGlGSLfWxvCbwUOKL0OUfEBCeJqVNab+OprRtYkqYAz6GaBfxZtJ5D8Sxgl6HkVVq/awFXSjpG0lxJCyU91f0g6WZJR0iaD+wpabakPcqxbSRdIWmepAslrVmOzQBOLa3SXSSd11DeaySd2yLOFan6mZ9opx4iov7qNk62aZKV9HJJ3wBuAw4FrgQ2sv3GZtfZvge4Cti57JoJnFHWCvtEmZF8c+AVkjZvuPRu21vbPq0hhqWBo4E9bG8DnAh8wfZZwFyebpX+ANhY0vRy6f7l3JGcKunXwI3A58pEOMM/+4HlH4O5dy26q9nHjYgamdLGNt7xjEjSrcAXgeuALW2/zvZJtu9ts+zGLoPGroI3l9bqr4BNgU0arjl9hHI2AjYDLpa0APgksM7wk0oCPxnYV9IqwHbABaPEto/tzYF1gUMlPX+E8mbZnmF7xvRp059ZQkTUjqhfS7bZ6IKX2b5lCcr+DvA1SVsDK9ieJ2l9qhbxtrb/Kmk2sFzDNQ+MUI6o5rTdro33PAk4H3gYONP2481Otn1XSfj/TDUBb0RMcDV74Kvp6IIlSjq2/w5cRvUn+1ArdmWqRHqvpDV4ujuhmRuB6ZK2g6r7QNKm5dj9wFMrN9i+HbidqrV7UquCJa0AbAX8oZ3PFBH1JtG1G1+SPljuHV0raY6k5Vpf9Uy97p6YA2xRvmL7GqpughuAbwOtRilg+1FgD6pRANcAC4CXlMOzgWPLja/ly75TgVttX9+k2FNL18M8YLbteWP7WBFRV1PUemtF0trA+4EZtjcDpvLMEVNtaWu12k7ZPo9hjwrb3m+Uc9cb7TzbC4DtR7jmbODsYbtfBhzXJKYdmoQcERNcF7tclwKWl/QYsALVX8kdFTIiSUcDo64pbvv9nbxhL0maR9UdcUi/Y4mI8VetjNBWlp0maW7D61m2Zw29sP0nSV8B/gg8BFxk+6JOYmrWkp3b5FgtlSFeETGJtdkHuqgMJR2RpFWB3YD1gb8BZ0ra1/YpY42n2ZLg3xxrYRER/dal7oJXAzfZvqsqU+dQ3QvqXpIdUgb3/wfVeNan7q7ZfuVY3ywiopeGHqvtgj8CLy4jkB6iWga8o7/u22lZnwpcT9VsPhy4Gbi6kzeLiOi1bowusP1LqukB5gO/ocqVs5peNIp2RhesbvsESQeXRRWvkJQkGxG1M4YbXy3Z/jTw6SUtp50k+1j5eoekXaiGMay2pG8cEdELNZtOtq0k+3lJz6YaFnU01VNbH+xpVBERnWizO2A8tUyytr9Xvr0X2LG34URELBmN+1KJzbUzuuAkRngowfY7ehJRRESHBCxVs2Ve2+ku+F7D98sBb6TDx8siInqtbmt8tdNdsNjcAJLmAD/pWUQRER2qRhf0O4rFdTJBzAZUS9FERNRLHxZKbKWdPtn7WbxP9s9UT4BFRNROt8bJdks73QUrtTonIqIOBEyt2Y2vluFIurSdfRER/SemtLGNp2bzyS5HNVHttDLt11BkKwNrj0NsERFjUi2k2O8oFtesu+BdwAeAtaiWaRkK/T7g670NKyKiAxPpiS/bRwFHSTrI9tHjGFNERMfqduOrnS7iJyWtMvRC0qqS3tu7kCIiOjPUXdBqG0/tJNl32v7b0AvbfwXe2bOIIiKWQLeWBO+Wdh5GmCpJtg0gaSqwTG/DiogYO9H2Gl/jpp0k+0PgdEn/W16/q+yLiKgXTcC5C6ie7joQeE95fTFwXM8iiohYAt1KseVe1PHAZlRPvb7D9s/HWk47T3w9CRxbNiS9nGry7n8f65tFRPRSN5efAY4Cfmh7D0nLUD03MGZtTRAjaStgb+DNwE3AOZ28WUREr3UjxZbVYLYH9gOw/SjwaCdlNXvia0OqxLo3sAg4HZDtrI4QETUlprQ3emCapMYlvmfZblyNdn3gLuAkSVtQPZB1sO0HxhpRs5bsDcCVwK62fw8gKWt7RURtjWF0wSLbM5ocXwrYGjjI9i8lHQV8FPjUWGNqFs+bgDuAyyQdJ+lVdK9POSKiJyS13NpwG3Cb7V+W12dRJd0xGzXJ2j7P9kxgY+AyqnkMniPpGEmv7eTNIiJ6TW1srdj+M3CrpI3KrlcB13USTzujCx4Avg18u8zGtSfVsK6LOnnDGDy3Xnlkv0OovQPm/KrfIUwO3R0nexBwahlZ8H/A/p0UMqblZ8ojtbPKFhFRKwKmdinJ2l4ANOu3bUsna3xFRNRW3W4cJclGxECp2VO1SbIRMTiqIVz1yrJJshExUNKSjYjoGaG0ZCMieqObowu6JUk2IgZHH5aXaSVJNiIGSpJsREQPpU82IqJHqkm7+x3F4pJkI2KgpCUbEdFDXVx+piuSZCNiYKS7ICKip/IwQkRE72ScbEREb9UsxybJRsTg6PZjtZKmAnOBP9netZMy2lzYMSJigujGIl9POxi4fknCSZKNiIGiNv5rqxxpHWAX4PgliSfdBRExULrYW3Ak8BFgpSUpJC3ZiBgobfYWTJM0t2E7cLEypF2BO23PW9J40pKNiMHSXkt2ke1mK9G+FHiDpNcDywErSzrF9r5jDSct2YgYGFL1WG2rrRXbH7O9ju31gJnAjzpJsJCWbEQMmIyTjYjopS5nWduXA5d3en2SbEQMkMxdEBHRU5m7ICKiR8b+QFfvJclGxEBRzZqySbIRMVBqlmOTZCNisNQsx/bmYQRJl0l63bB9H5B0zBjK+KykV7c4Zz9Ja40xtqUlfUnS7yTNl/RzSTuPpYyIqKl2nqkd5yzcqye+5lA9JdFoZtnfkqSptg+zfUmLU/cDxpRkgc8BawKb2d4a2J0lnAAiIuqjW7NwdUuvkuxZwC6SlgGQtB5VMrxS0jFlQoaFkg4fukDSzZKOkDQf2FPSbEl7lGPbSLpC0jxJF0pasxybAZwqaYGkXSSd11DeaySd2xiUpBWAdwIH2X4EwPZfbJ/Ro3qIiHE0tJBiq2089STJ2r4HuAoY+jN8JnCGbQOfKBMzbA68QtLmDZfebXtr26cN7ZC0NHA0sIftbYATgS/YPotqxvJ9bG8J/ADYWNL0cun+5dxGLwT+aPu+Ln7ciKiTSdJdAIt3GTR2Fby5tFZ/BWwKbNJwzekjlLMRsBlwsaQFwCeBdYafVBL4ycC+klYBtgMu6DR4SQcOTYN216K7Oi0mIsZZ3boLejm64DvA1yRtDaxge56k9YFDgW1t/1XSbKppxIY8MEI5Ahba3q6N9zwJOB94GDjT9uPDjv8eWFfSyq1as7ZnAbMAttlmhtt474iogboN4epZS9b234HLqP5kH2rFrkyVSO+VtAZPdyc0cyMwXdJ28NTogE3LsftpuGll+3bgdqrW7kkjxPQgcAJwVEN/8XRJe479E0ZEHdWst6Dn88nOAbYoX7F9DVU3wQ3At4GftirA9qPAHsARkq4BFgAvKYdnA8eWG1/Ll32nArfaHm3xs08CdwHXSboW+B6QPtqIQVGzLNvThxFsn8ewj2R7v1HOXW+082wvALYf4ZqzgbOH7X4ZcFyTmB6lWrfnI6NHHhET0dCk3XUyUE98SZpH1R1xSL9jiYj+qFeKHbAkW4Z4RcRkVrMsmzW+ImKAtDOAq3UWlvS8Mj3AdeXBqYM7jWigWrIREV3qkn0cOMT2fEkrAfMkXWz7urEWlJZsRAwMUSXZVlsrtu+wPb98fz9wPbB2JzGlJRsRA6XNJ7qmSZrb8HpWeQDpmeVVc69sBfyyk3iSZCNioLTZXbCozKHSoiytSDVM9AOdznmSJBsRA6VbgwvK5FRnA6faPqfTcpJkI2JwtNnn2rKYaqGwE4DrbX91ScrKja+IGDBdea72pcBbgVeWx/YXSHp9J9GkJRsRA2No0u4lZfsndKnnIUk2IgZKzaYuSJKNiMEy3pNyt5IkGxGDpV45Nkk2IgZLzXJskmxEDI52H5sdT0myETFQVLMsmyQbEQOlXik2STYiBkzNGrJJshExSNqblHs8JclGxMAYmk+2TpJkI2KgJMlGRPRQugsiInol42QjInqn7YkMx1GSbEQMlppl2STZiBgo6ZONiOihbkza3U1ZfiYiBktXVp8BSTtJulHS7yV9tNNwkmQjYqCojf9aliFNBf4H2BnYBNhb0iadxJMkGxEDY+iJr1ZbG14E/N72/9l+FDgN2K2TmNIn24b58+ctWn5p3dLvOIaZBizqdxA1lvpprW519PwlLWD+/HkXLr+0prVx6nKS5ja8nmV7VsPrtYFbG17fBvxzJzElybbB9vR+xzCcpLm2Z/Q7jrpK/bQ2iHVke6d+xzBcugsiIp7pT8DzGl6vU/aNWZJsRMQzXQ1sIGl9ScsAM4HvdlJQugsmrlmtT5nUUj+tpY5GYftxSe8DLgSmAifaXthJWbLd1eAiIuJp6S6IiOihJNmIiB5Kko0oVLe1pGtC0nLl5k90IEk2JjVJ0yX9E4BtS8rvRANJGwFHA3tKWrnf8UxE+YEaUJI2lPRuSZsN25//54WkfwR+BHxV0nwA20+mRVspz+p/C7gKuMj2fX0OaULKL9zgehfwDeBMSQdL2gKqJNLfsOqhtNBmA1+2/RrgLkmvkjTFGXKDpBWBrwBft32c7bvK/uSMMUqFDa5jgaOAQ4DNgXdJ+pak1SQtB5P3F0bS8lQzLF1r++Sy+x+AdwMXSXp9+iB5BLgf+IWkKUM/K0P/SEt6Vj+Dm0gm5S/ZJHEr8AJgE9sHAN8E9qVq3R4racZkbdXafoiqHtaRtJek7wI/tL0ncClwMNXkKZNS6S5ZEVgVWL3x50SVZwG7SVq2XzFOJEmyA0iSbD8MfBB4saQ3AceU118AbgQm5U2Mof5W2+dQtfY/VL30QWX/F4FHga37FmSfufJX4IfALEkblL7qZUpXyj9TzbO6XF8DnSDyWO0AKcnV5S65gDuAm6gS7Mdtn1BO/U3j+X0Kt29Kv+uTts+V9DDwYUlvLK+3Al5INbXdpFN+blRar0cCKwHflnQAcLuk9cv+j9u+t2+BTiB5rHaAlJbGo43JU9KrgBOBzW3fO5Rg+htp/zTU0VTbT5R9b6Lqj/0Z1cTMn7L9vX7G2S/Df4ZK3/QHgTcBtwOrA18r/yBNyn+kxypJdkBI2gD4BPBe2w8OO3YUcA/wxTLL+6Q0vI6G/WP0JuCTwCdsXzAZE0iLn6G1gQeA5W3fMRnrp1PpLpjgGn7Yl6LqS5wybD/AT4EbJmuCHa2OGo/ZPkfS5bbvmWwJpMnP0BSe6qL10Fyqf6Ps7EesE1FufE18q5avN5bvPw+L/xLYPsP2r/sQW12MWkcN/dfYvmdof1+i7J/R6ufJSVgXXZckO8GUMYsq368NnCTpveXw+6vdWr9vAdbAWOtosiWS/AyNr3QXTCDlIYIdgCsl7Ug1zOi/gEOBDaiWy1gW2BS4abL92Qupo1ZSP+MvN74mGEmHAntSDa15v+1LJK1CNWbxIGAnqn61PRr60SaV1FFzqZ/xlSQ7QTQMqXkO8D2qO717APc33tBSNenJu4CTbc/rT7T9kTpqLvXTH+mTnQAafjmeCzwM7ApcQvWQwQvLOesA2L6eamXN1/Up3L5IHTWX+umfJNmaa/jl2Bk4B/gPqrGcX6AaHP6xctPiKkmbSlqB6pHZ7/Qv6vGVOmou9dNf6S6oqcYnsyRtA5wAvAV4M1UL4yXlF+cg4PnAFbbPL+cvMxnGxKaOmkv91EOSbA1JWgvYBrjA1dLE21Ld+f0z8CVgL9s3SdrM9rWSlirnPTXZ9KDfEU4dNZf6qY8k2RqStANwF/AnYGmqafeuAO6lmoPgoTL85h3AB20v6lOofZM6ai71Ux/pk60h25cDf6WadHsP4Gbgo8AiYDtJu1LNhHTmZP3lSB01l/qpj7Rka0jSa6nu+D4I7AhcDlwGbAv8G9WffGfY/v5kHSyeOmou9VMfeeKrZiRtTjVQ/Hjbv5R0L9U0c6Yat3hmw93iSfnLkTpqLvVTL0myNSFpKrACcDbwF+AWAFfzdj4B7AcsJelk24+UY5PqlyN11Fzqp57SJ9tHKgBsP2H7fqrhNatQtUQox74LnAxcPfTLMVmkjppL/dRf+mRrQNLLgRcBc21fUf7cOxv4qu1j+htdPaSOmkv91Fdasn0gaa3yS4GknYBZVDcojpP0ceC3wBuBw8pA8UknddRc6mfiSJIdZ6XfbG/gdZKmA+8D/oVqwuQpwD9RPfZ4HfBayqKHk0nqqLnUzwRjO9s4b1S/ENcAq1E9I74ZcBXVTYtXA3dTjWmcWs5Xv2NOHdVrS/1MnC0t2T5w9Xz4ZcBnqaabm0Y13dyDVMt4zwXOd1lN1eW3ZDJJHTWX+pk4kmTHgaR1Je2sajXQIWdStUBWBK4EnpB0IdXNiq/ZXtiHUPsmddRc6mfiyuiCcSBpa+BjVK2N06kGhD8g6WzgL7bfK2lVYGfgJts/72O4fZE6ai71M3ElyY4TVWsrbQd8jupGxB+oWhxHAh+2/dv+RVcPqaPmUj8TU5LsOJO0BrAJ8GFgdaoF6z5o+7i+BlYjqaPmUj8TSx6rHWe2/0L1yONlknYDdqeanT6K1FFzqZ+JJS3ZPtDiM9ZPtf1EJupYXOqoudTPxJEkGxHRQxnCFRHRQ0myERE9lCQbEdFDSbIRET2UJBsR0UNJsjEmkp6QtEDStZLOlLTCEpQ1W9Ie5fvjJW3S5NwdJL2kg/e4WdK0YftOkvSuYft2l3TBGMp9t6S3tThn92afKSaHJNkYq4dsb2l7M+BR4N2NByV19ICL7X+zfV2TU3YAxpxkRzEHmDls38yyvyVJS9k+1va3Wpy6O9WTWTGJJcnGkrgSeGFpZV4p6bvAdZKmSvpPSVdL+vVQq7EsR/V1STdKugR4zlBBki6XNKN8v5Ok+ZKukXSppPWokvkHSyv65ZKmSzq7vMfVkl5arl1d0kWSFko6HtAIcV8KbCxpzXLNs6jmYD1P0mGlvGslzZKq9bNKfEdKmgscLOkzkg4tx14g6YeS5pV62Li0ut8A/GeJ+QWS5jd83g0aX8fgSpKNjpQW6848Pev+1sDBtjcEDgDutb0tsC3wTknrUy2HshFV6+5tjNAyLTP9Hwf8q+0tgD1t3wwcSzV935a2rwSOKq+3Bf4VOL4U8WngJ7Y3Bc4F1h3+HmWO1bOpFhyEagLsy23fB3zd9ralpb48sGvDpcvYnmH7v4YVOQs4yPY2wKHAN2z/DPgu1cQtW9r+A3CvpC3LNfsDJ41YuTFQMndBjNXykhaU768ETqBKllfZvqnsfy2w+VB/K/BsYANge2BOSXK3S/rRCOW/GPjxUFm27xkljlcDm5SGJsDKklYs7/Gmcu33Jf11lOvnAF+hStYzqVZyBdhR0keoVhhYDVgInF+OnT68kPKeLwHObIhl2VHe83hgf0kfAvaiWvgwBlySbIzVQ7a3bNxRkssDjbuoWnYXDjvv9V2MYwrwYtsPjxBLO34GrClpC6okObNMJfgNYIbtWyV9Bliu4ZoHnlkMU4C/Da+TUZxN1dL+ETDP9t3tBhsTV7oLohcuBN4jaWkASRuWfs8fA3uVPts1gR1HuPYXwPalewFJq5X99wMrNZx3EfDUKqwNf4b/GHhL2bczsOpIAZaJVE4HvglcUJL1UEJdVFqoe4x07bBy7gNukrRneU+VxP2MmMt7XAgcQ7oKJo0k2eiF46lWSp0v6Vrgf6n+ajoX+F059i3gGbP3274LOBA4R9I1PP0n+vnAG4dufAHvB2aUG2vX8fQoh8OpkvRCqm6DPzaJcw6wRfmK7b9R9QdfS5UMr27z8+4DHFDiXQjsVvafBnxY0q8kvaDsOxV4kuofiZgEMgtXxDgqIxKebftT/Y4lxkf6ZCPGiaRzgRcAr+x3LDF+0pKNiOih9MlGRPRQkmxERA8lyUZE9FCSbEREDyXJRkT00P8Hfpio6dL937wAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn. metrics import accuracy_score, precision_score, recall_score, confusion_matrix\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 从测试数据中获取预测集\n",
    "predictions = model.predict(X_test)\n",
    "\n",
    "# 获取指标\n",
    "print(\"Overall Accuracy:\", accuracy_score(y_test, predictions))\n",
    "print(\"Overall Precision:\", precision_score(\n",
    "    y_test, predictions, average='macro'))\n",
    "print(\"Overall Recall:\", recall_score(y_test, predictions, average='macro'))\n",
    "\n",
    "# 绘制混淆矩阵\n",
    "cm = confusion_matrix(y_test, predictions)\n",
    "classes = ['Variety A', 'Variety B', 'Variety C']\n",
    "plt.imshow(cm, interpolation=\"nearest\", cmap=plt.cm.Blues)\n",
    "plt.colorbar()\n",
    "tick_marks = np.arange(len(classes))\n",
    "plt.xticks(tick_marks, classes, rotation=45)\n",
    "plt.yticks(tick_marks, classes)\n",
    "plt.title('Confusion Matrix')\n",
    "plt.xlabel(\"Predicted Variety\")\n",
    "plt.ylabel(\"Actual Variety\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "分类准确率  0.9555555555555556\n",
      "混淆矩阵 \n",
      " [[16  0  0]\n",
      " [ 0 17  1]\n",
      " [ 0  1 10]]\n",
      "分类结果报告 \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "     class 0       1.00      1.00      1.00        16\n",
      "     class 1       0.94      0.94      0.94        18\n",
      "     class 2       0.91      0.91      0.91        11\n",
      "\n",
      "    accuracy                           0.96        45\n",
      "   macro avg       0.95      0.95      0.95        45\n",
      "weighted avg       0.96      0.96      0.96        45\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "\n",
    "#F1 = 2 * (precision * recall) / (precision + recall)\n",
    "accuracy = metrics.accuracy_score(y_test, predictions)\n",
    "confusionmatrix = metrics.confusion_matrix(y_test, predictions)\n",
    "target_names = ['class 0', 'class 1', 'class 2']\n",
    "classifyreport = metrics.classification_report(\n",
    "    y_test, predictions, target_names=target_names)\n",
    "\n",
    "print('分类准确率 ', accuracy)  # 混淆矩阵对角线元素之和/所有元素之和\n",
    "print('混淆矩阵 \\n', confusionmatrix)\n",
    "print('分类结果报告 \\n', classifyreport)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优缺点\n",
    "优点\n",
    "- 直接、快速\n",
    "- 知名度高\n",
    "\n",
    "缺点\n",
    "- 要求严格的假设\n",
    "- 需要处理异常值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 决策树"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练和分类\n",
    "\n",
    "从sklearn中导入决策树分类器，对数据集进行训练和分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DecisionTreeClassifier()\n",
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n",
      "精度是:1.000\n"
     ]
    }
   ],
   "source": [
    "from sklearn import tree\n",
    "from sklearn.tree import DecisionTreeClassifier  # 导入决策树DTC包\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import load_iris  # 导入方法类\n",
    "\n",
    "iris = load_iris()  # 导入数据集iris\n",
    "iris_feature = iris.data  # 特征数据\n",
    "iris_target = iris.target  # 分类数据\n",
    "\n",
    "clf = DecisionTreeClassifier()      # 所以参数均置为默认状态\n",
    "clf.fit(iris.data, iris.target)     # 使用训练集训练模型\n",
    "print(clf)\n",
    "predicted = clf.predict(iris.data)  # 使用模型对测试集进行预测\n",
    "print(predicted)\n",
    "print(\"精度是:{:.3f}\".format(clf.score(iris.data, iris.target)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEDCAYAAAD3FPVyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtUUlEQVR4nO3deZgU5bX48e/pYRkGWRRFEAOjxC1uRNSYiIqCuCWKXre4xGueSERjcMm9yS8aMsYFE68iYkxCTC4u5KrXfbshQYVIDLmAEjUKiXgBR6Ig0UHBmZGZ8/vjrZ7pnumerpmprnmr+3yep5+hqqvfequGOf32W6dOi6pijDHGT6me7oAxxpj8LEgbY4zHLEgbY4zHLEgbY4zHLEgbY4zHLEibsiUikmNdr57oSz4isnNM+2l3LowfLEibLCIyT0ROyli+WERuzbFdRb6AJk7fEPu6VkRO7+D5/2zTFxGRp0VkrxzbfktEKkXkORHZX0T+TUQGisjPReTIPLs4TER+32bdIhE5qIM+DRCRrwX//nzQ/qCOjzQ8EblARH6Tser/Qrxm7+Dn7iJyTAfb7SsiFwb/PkNEfhn8+yvAnd3ruSkWC9IJJyI1IvKxiHwoIs+LyH4i8h0ReVdEPhGRuuDfRwfbHysi/yci60Tk0ox2zhORe4EvANNEZK6IzAXOAyaKyN0i8tWMXZ8MrBKRlcHjQxH5h4isBFYGz/UN2h4pIo/kCNw7Af3zHNeFwNnAj0TkzyLyGPANYDzwGxH5k4i8lBHkewHXANuCNs9W1c3ABKA2z+nbH/hjxj4HAzsCL+fZHuAT4Jsicj7wFrAVOEtErhKRX4jI/kFb3xSR90RkWZ7HJhGZFGybEpE+QfuNwMcZ+/s0o3/92r4xisiJwMPBSFiBOSJSlafvW4HrRWRc0G5DsO1PgCc6OGbTg8RuZkk2EakBKoEfAN8DLgL2UNWGIMguUdWfB9tujwugXwl+vgR8WVVXisiOuOB2M/AIsDjYxRlANfAfwFZV3ZSnH3cBi1V1bo7nUsBTwAeqeq6IPATsAQzHBaR/AlXAEaq6SUSOBWbiAvKNwP1BXxcBxwK3AfNU9emg/Urcm8Zw3JvKw8DOwKPAA6o6PNiur6o2BP9+AjgUF3TrgOuBgcAM4KOg68NVtb+IfCE4D5W0vgl8Cdge+G1wLl8H/g6sCc79+cCtwDu5zhcwEjhLVX8vImOBnwMNuDeJQcDqYLvDgCXBv/sA31LV/804r0uAG1T18WDdLKCfqk5p8zvoiwvMpwGfDfo8EXgoOLZvAX3S58f4w6v5N9N1qvopcF0wAj0aFzzaOgVYmvFHviDYdqWqvg+8LyJbgWtpDVRDcAHx7W70rVlEzgH+JCJ7qerpwf5/BixU1QdEZA1uJAiwALgMF3TnAxIs3wsMxQXtj4NpiddwwetIXNA9GHgDF/inAikRWYYLih+JyBhV/QgXqPZW1Q9F5HpgAHAB7o3ib0H/0oHyNeDvqvrPYP0UXFDdP3isBRYC1wEPAn8KjuU+Vb0ifR5EZCfcm9P+wCpVXRicn+XAIcE2ZwGfV9XvBW+cS1V1XJ5T+x2gLh2gA1cDS0TkFuA72joKexz3pvIp7s1of+AfwOHAFtybcl8RmaiqH+bZn+kBFqRLz1+AvckdpPcDVmUs3wg05djuJVzgAffHnEVEegOqqts66kjwEb5JVZuCYPh5Va3P2KQ3bvSY1kzQcDC6bMIFu8/ggu8nuJFsWl/c6PUT3NTDCKAe+BD4ADfavUNVrxOR+4FbggDdsq8MRwW7/luO/mwBtojIT2h9I3gdN30AbiQ6Otjn0vThA80i8iTuzaYZdy7vDfo6IMf5Etwb6ZvBqn/H/Y4QkdOAFar6VrA8AbgcF2RbqOrHwbz074E/iMg3VHWVqh4fvG4vYC7uTaYJeBc4ArhcVd9o2yfT8yxIl56Pge3yPDeYjI/fqroOWj4291LVxuCphcCfg39vxY1UW4IucBVwkYi0zJcCw4CTROR7Get6Ad8Eng2WR4nIm0BFsK/+uKCaViEiFaraBNyNm65IG4EL0LtmrDtHVT8Skd2A93FB7UfAD4HJuDeBscG2I2mdQsjlf4G7ROQuVf1GsK5tIN8bN3rdHmhU1UXBeVkJzMgYtYKbvmkEzscF5B2B3+E+JQwHjhCRF9Kj1uDc3gXsqarnBG00Ap+KyNm4YH028JaIDADm4Ka4/iQi/XBvCpuD1+0AXIF7M2kK2v8C8HXcSH4KsC8wQVWvF5HxwOMiMkdV/6ODc2R6gAXp0tOf7AtPmT7FjT6BltHZJ8AG4Oci0oALhsfRGtCGA5XBH3If3IjrJuCmzIY7mpMOnk8BTwI/A74e7OtzwCQR+T9ckH0OF2AfC162naqOCUaYi4H7VXV20N78jObTQela3MXwJ4Nj/RKwMLggOCA9XZHhBRFpwr3BfE9VF4vIzSJynKrOp710EH4PN7W0BvglUNsmQIMLzB/jph/2wJ3jvrg3i3/gPtFsDY5lN+A3uNF529/dt3DB94uq+h5A8Ma0T/BG96tghP+Oqs4K2vst8Iaq/iKjnSrgIVX9fbDNYII3c1VdKCKH4n7XxjMWpEvP/rhAmMubuOyNtBOBl1T1f4I/0tOB7wNnBv9OAXcAT+PmWue0ma7ojDOAd1V1JjBTREYAy3Aj6Uk5AmjvjH//ENgdmC4iz+LmjvcFNoKb0w0yFhbjLrQdB5yoqp+IyCO4+djncvTpiIw56bT/AL4mIotoHzB74Uah63EZIBfipls2Bhdw16rqfwbb7gKsCfY7JHh8FTcXfhjuwuYk4Erc9Mz1uMB9VzCqPic4zqeAS1U1a1Sf8akH3FTNtIzlEbTPaJmNm7L5YbBcDQwVkfQFYgG2E5GrVHVB2xNleo4F6RIRpGZ9B/fHtjDPZo8AVwdpYhuA44Fbg4/C9wErcIHiv3FBBVxwnQ/sA/xDRI5V1WWd7NtAXNbIORmrZwG34Eac94jI5DZz3O8BJwQXwD4bbN8f+C9c4NpfVT/J2H4XYDluLn5vYHrGMd+Eu6gXxuNB+4NpnT5Iu5jcKYOCe/M7EUgH6b2Bv+GO+++4gN2AGzHX4j45fAKgqh8AT4vIZ4PXfg8YhZtX/nM6QIvIUFXdkLVjkVNw00dLMlYPp01Wiarul/GaHXGph/cB61X1mnwnw3hAVe2R4AdQgxvxfQD8DzA647m5wMVttj8RF4jfBqYG6/rgMh3A/cGDC/iX40bTlcG6PQjSNjPa6weMwQXHC/L08XDcR22C9mbhRpgVuBHzQlx63X4ZrzkCdxHurmCbf8OlFw4Kjuu94Oe+wfaCS897ETf3ehsu3ewl3Bz6W8AXMtpfBbyKe2N6F7goo3+VuFHsL4N1u+IyNn4fHGf6MR83er8IN0r+cbB936B//dqch01tlh8HvpSxvAewKGP5elyGRvp3tBQYn/H8KbhgfEiwPBA3xVOb5/dQgUu//GvwM4VLdZyf2Q97+PXo8Q7Yw48H7qLUiiDoLA6C2Krg3y/iskZOz/G6CtzH/5UEgb6DfYzEjeDm4+ab0+srcdMqjcCpuDnm+cAZGdtcg/vYn14+EPgx7tNgFfB8EHCGBc+fi7tIt2ewPB534a4qWP4G0DejrfR2w3Gj738HBoY8d9fhRscTg+UzgAU5tnsxCJDLgnO9Auif8fx+uLz29PKk4M1hWRCg76b1TfQ2XGrg2Izt/z1Y97U2+00Fr12Dyy7Zo83zk4J9rAZ27un/i/bIftjNLCY2wZTMqcDD2maONXh+BO7jd6L+U4rIMKBB3bRFet0IVc13I0sU+xwA1KvLjw+z/edwN9ps7WCbalVdE1EXTUQsSBtjjMesdocxxnjMgrQxxngs0hS8HXfcUaurq6Ns0hhjSt7y5cvfV9Wdcj0XaZCurq5m2bJOpdAaY0zZE5G1+Z6z6Q5jjPGYBWljjPGYBWljjPFY0Wt3fPrpp9TW1lJf39W6PKarKisr2XXXXendu3fhjY0xXip6kK6trWXAgAFUV1cj9oXEsVFVNm3aRG1tLbvttltPd8cY00VFn+6or69nyJAhFqBjJiIMGTKkZz/BzJsH1dWQSrmf8+b1XF+MSahYSpVagO4ZPXre582DKVNga1AqYu1atwxw7rk91y9jEqbsLhxmVP7KqbnZ1f2pq6tj48aNcXWr9Fx9dWuATtu61a03xoRWdkX/77jjDiorK7noootyPj9t2jSOPfZYRo8ezcyZM7n22mv5+9//zsiRI9l999356le/ytq1a6mqqsp6XV1dHRMnTmTGjBlxHIb/1q3r3HpjTE5lF6R79+7dLtvhjTfeYNmyZbz77ruoKpdddhk777wz2223Hddffz1jxozhM5/5DAD9+vXj17/+NXvvvXdWG4sXL+Z3v/tdbMfhvZEj3RRHrvXGmND8C9Lz5rmPxOvWuT/oG27o9hzmZZddxuzZs3M+9/3vf59LL72UAw88kC9/+cvU1tZy+umn89e//pVt27Zxwgkn8M4777Bq1SpGjx6NiHDBBRfQr1+/dm2dc845OfZQpm64IXtOGqCqyq03xoTmV5Au0sWm559/Pu9zTz/9NDfeeCODBw/mpJNOYt26dVRWVjJ06FC2bdvG/Pnz+dKXvsQ3v/lNAJqampgxYwaffPIJGzdu5O2332blypXU1dVxwQUXdLmPJSf9+4r4DdeYcuNXkO7oYlM3/rh79cp/mOkMiFQqxXbbbcezzz7Lr371K6644goWL17Myy+/zDXXtH5P59atW2loaGDDhg0MGzaMOXPm8OCDD7Jlyxbq6uoYOnRol/tZcs4914KyMd3kV5DuwYtNqkp9fT29e/fm0EMPpba2lkcffZSLL76Y2tpadt11VwA2btxI3759eeqpp+jduzerV6/m2muvZejQoZx11lkWpI0xkfIrBS/fRaVuXmwK8xVhFRUVTJ48mccee4x169bxwgsvcPDBB7Ns2TKOPPLIlu3effddhgwZwvnnn8+kSZPo168fvXr14s033+TOO+/sVj+NMaYtv0bSRbrYtHbtWsaPHw/A+vXrSaVSzJ07F4DVq1fz0ksvcfnll2el1TU3N7NlyxYeeughHn/8cQA2bNjAoEGDePfdd2lsbGT06NEMHz6cGTNmMHjwYJYsWcLq1asZPXp0t/prjDFpfgXpIl1s+vDDD/M+t99++7H//vvzhz/8od1z9fX1nHLKKRx66KEAPPzwwxx33HEcd9xxLduoKqeddhqpVIq1a9e2BHRjjIlCpN8WfvDBB2vbb2Z544032GeffSLbR9zq6+uprKwEXECuq6tj8ODBPdupTkj6+TemHIjIclU9ONdzfs1JeygdoMFlgiQpQBtjks+CtDHGeMyCtDHGeMyCtDHGeKwsg/R7771HY2Nj3uc/+uijUO2UdFlTK9hvjBf8SsErknQwTaXce9KsWbM44IADOPvsswHYtm0bqVSq5fmTTz6ZH//4xy2pd/mUbFlTK9hvjDe8C9KqkPmFIm2Xu+LFF1/kBz/4ARUVFS0FlJ566immT5/OyJEj+fTTT5k+fToTJkxg9uzZbN68me9///s0NzezdOlSVqxY0XKDSlmUNS1SDRVjTOd5FaRrauDDD2HmTBeYVeGKK2DwYPdcV40bN46bb76ZRx99lOOPP57q6moA1qxZw4YNG/jud7/LDjvswK233sry5ctZunQpqVSK2bNnc/LJJ2fdQThw4MDSL2tqBfuN8YY3QVrVBehZs9zyzJkuQM+aBdOmdW9EvXHjRqZOnco999zD008/HezP3cTzxS9+kWOPPZbZs2fz2muv8cwzzzBp0iQ++OADPvjgA3bffXduv/12fvrTn3LCCScwYsSI0i9ragX7jfGGN0FaxAVmcIE5HaynTWsdWXdVZWUlP/3pT9l777155ZVXGDp0KE1NTQCceuqpDB8+nEMPPZSxY8fy1ltvsWDBAhYuXMjChQupqamhpqYm66aWki9ragX7jfGGN0EaWgN1OkBD9wM0wIABA1i/fj0HHXQQ77//PnvttRcAb7/9NrfffjuLFi1quWi4YsWKlmJMAOPHj2fNmjUcfvjhLetKvqypFew3xhteBen0HHSmK66IJlBPnjyZr3zlK5x77rl8+9vfZtiwYXz961/n/vvvbwnQAGPGjGHhwoVZr33ttdfYfvvtW5Yzy5oCrFu3rqWs6TnnnMNbb70FZJc1/ec//8mSJUuyypr+/Oc/795BFZMV7DfGC94E6XSATs9BZ85JQzSBuqKigttvv51x48axZcsWHn30UYYNG9byvIhQX1/f7nX77bcfjzzyCIcffjjvvPNOz5c13bQJ3nkHGhuhTx8YMQKGDOl8O8YY73kTpEVcFkfmHHR6jnrw4O4F6PXr1zN9+nRqa2sZNGgQt912G1VVVdxyyy28/vrrbL/99jz44IMMGzaMPn36cMQRR9C3b9+Mvgn9+vXjpJNOYuedd+7ZsqabNrmLekHuN42NrRf5LFAbU3K8K1VajDzpdD/22muvrKkN176yfv16RowY0e19xFLW9JVXXGBuq08fOOCAdqutVKkx/uuoVKk3I+m0tgE5igAN5A1UIhJJgIaYyprmu529g9vcjTHJVZa1OxKtT5/OrTfGJJoF6aQZMcIVPcqUSrn1xpiS4910hykgfXHQsjuMKQtlF6TffPNNdtllF5qamqivr2ennXbq6S61aG5uJpVKUVdXR2NjY/6+DRliQdmYMlF2Qfrb3/42N910E6+++ip/+ctf+MlPfpJzu7vuuovGxkYuueSSnM/379+fz3/+81nrtmzZwp577skDDzwAwDHHHAO0Bt/0T1Vln3324c4778x6fcmWPs1n3jy7q9GYArwK0gNnDOSjxvYF9wf0GcDm/7e5W22PGzeO/v37s2LFCi677DI2b97Mxx9/zCuvvELfvn254IILmDVrVkvFunfeeYfm5maeeOIJwKXXTZ8+vSXw7r777ixevDhrHytWrODmm29uWX7uuecAqKmp4bDDDmPJkiUcdthhHH/88S3blEXp01ysZrUxoXgVpHMF6I7Wd0Y6oE6ePJnbbruNZcuWsXLlSq655hoOOOAATjvtNE477bSW7efOnUt9fT0XX3xxzvbWr1/PuHHjstalR9JpzzzzDMcee2y713788ccsXbqUo48+ujxKn+ZiNauNCcWrIF0sGzZs4KyzzqKiooJXXnmF8847j82bN7N161YWLlzY8nVZd9xxB6+//jqpVIqVK1fS1NTEa6+9BsAhhxySVV50+PDhLF68mIaGhqy7EzP99re/pa6urt36hx56iJUrV3L00UeXR+nTXKxmtTGhlEWQ/uijj2hoaODFF1/kxBNPZObMmbz66qstI+m0888/H1WlV6/s06KqSMZdNQ0NDS03ruy7776sWrWKiooKNmzYwAsvvMC//Mu/APCtb32LKVOmZFXVA/jFL37Bk08+2bJc8qVPc7Ga1caEUhZBur6+nvfff5+JEyeybNkypkyZwgcffNAykk5/XdaBBx7IMccc067o0RtvvNFS2Q6gtraW4cOHA1BVVcVHH33EXXfdxb333svZZ59NU1MTFRUV7LnnnqRSKTZs2NDy2vnz53PAAQew4447tqwr+dKnuVjNamNCKYsgve+++/K3v/2N9evXc8IJJ7Bo0aKWKYdrrrmGq666iv79+9O7d2+OOuoo5s6dm/X6iRMnZk1pLF68mIMOOghwAfbCCy/kwgsv5KWXXqKioiLrtc888ww33XRTVltt57LLovRpW1az2phQvArSA/oMyJvd0V3r1q3jzDPPZPr06YALrqrKvHnzeOyxx7jyyitRVX73u98xceLErNe+/PLLWcu//vWvmT17dsvy3XffzcCBAwE3laGqTJkyBVXNqucBLiAPGjSo5RvKV6xY0fOlT3uK1aw2piCvgnR30+zyefnllznllFOYOXNmy3xxQ0MDDQ0NnHjiiUyePJn+/fuzevVqJk2alHMknb5A+Mwzz7D99ttzQFBx7sADD+Swww5jhx12AGD16tU88MADPPHEE9x44430CWpqLFiwoOXnddddR2NjIzfddBMTJkyIv/TpJZfAnDnQ1AQVFW7aoU3OtjHGD96VKi2WjrIwOquxsbEl+BZTFKVP253/Sy6Bn/2s/YZTp1qgNqaHdFSqNJYCS1G+EXRVVAEaiCVAQ/dLn+Y873Pm5N4433pjTI8qepCurKxk06ZNXgTqcqKqbNq0qd2cOMG3pLeTb70xpkcVfU561113pba2lo0bNxZ7V6aNysrKlvS9FhUVuQNym6wUY4wfih6ke/fuzW677Vbs3ZiwpkzJPSedrpthjPGKV9kdJgbpi4OW3WFMIliQLkd33mlB2ZiEsK/PMsYYj9lI2jcTJ8Kzz7YuT5gAwY0wiWNF/Y3HoqhfX8wa+Gk2kvZJ2wANbrnNbeqJkC7qv3YtqLYW9Z83r6d7ZgwQTf36YtbAT7Mg7ZO2AbrQep91VNTfGBOaBWlTHFbU35hIWJA2xZGveL8V9TemUyxI+2TChM6t99kNN7gi/pmsqL8xnWZB2icLFrQPyEnN7jj3XHfDzKhRIOJ+zplj2R3GG/nq1Hemfn0UbRRS9FKlxhhjOtbjpUpNJ8ybB9XVkEq5n11JWSvURhT7MMbEwm5m8Uk6tzidupbOLYbw0wSF2ohiH8aY2Nh0h0+qq13QbGvUKFizJpo2otiHMSZSNt2RFFHkFhdqw/KXjUkUC9I+iSK3uFAblr9sTKJYkPZJFLnFhdqw/GVjEsWCtE+iyC0u1IblLxuTKHbh0BhjephdOAR/coN96YcxRTJwxkDkWmn3GDhjYE93LZHKI0/al9xgX/phTBHFUWO5nJTHSNqX2sa+9MMYkxjlEaR9yQ32pR/GmMQojyDtS26wL/0wxiRGeQRpX3KDfemHMSYxyiNI+5Ib7Es/jCmiOGoslxPLkzbGmB5medI+ueQS6NXLjaR79XLLnXke4sm1tnxuY/ygqpE9xo4dq6YDU6eqQvvH1KnhnldVve8+1aqq7Oerqtz6qMSxD2NMC2CZ5omrBac7RORzwMlAn4zA/qNc29p0RwG9ekFTU/v1FRWwbVvh5yGeetBWc9qYWHU03RHmjsMHgRpgY5SdKku5AnDm+kLPQzy51pbPbYw3wgTp94BHVTVPBDGhVVTkHymHeR5cTnWuUW6UudZx7MMYE0reC4ci8jUR+RrwF+B5EZmasc50RbpOR771hZ6HeHKtLZ/bGG90lN0hweMvwK+ArcGy6ao774SpU7NHzlOnuvVhnod4cq0tn9sYb4S5cDhEVTdlLJ+pqg/m2tYuHBpjTOd1N0/6v9ssX9r9LhkTjbZjjAjvzTLGCx3NSR8lIj8EqkVkevD4MfBBfN2LUBQ3Z4S50aS7bYTpZ3ePJYrj8EBNDVxxRWtgVnXLNTXh2whToN6K2Jue1FF2xxpgITAZWBSs+wR4uag9KoYoiu1fcgn87Gety01NrcuZc8bdaSNMP7t7LFEchwdU4cMPYdYstzxzpgvQs2bBtGnueQlxBSVMgXorYm96Upg56W+r6u1hGvN2TjqKmzPC3GjS3TbC9LO7xxLFcXgiPXJOB2pwAXrmzHABGkCuzb+h/lBDb2NMd3Q0J503SHeUaqeq9+Ra722QTqVyT1aKQHNzuDY6+qsPOxFaqI0w/ezusURxHB5Jn7a05ubwARosSBs/dPXCYToF7xxgHFAJfBFIXp50FMX2M28oCbO+K22E6Wd3jyWK4/BEeiSdKXOO2phSkDdIq+rdqno30FtVp6jqHFWd2tFrvBXFzRlhbjTpbhth+tndY4niODyQOdUxbZobQU+b5pYtUJtSEua28DoRuRV4BfgckLyrJekLaldf7epPjBzpglpnbs5IX1SbM8fN6VZUuMDWmYtthdoI08/uHksUx+EBERg8OHsOeuZM99zgweGnPAb0GZDzAmBmgfow2xhTLGEuHPYCTgN2A9YCj6hqY65tvZ2TNiWrbRZH2KwOX/djylO3quCp6jZcJTwThXnzOh4FF3reZGkbKIsROPvWDKRR2o+k++gAGmo2R7/DHOziZfkKM91holIoxzmKfG4TKVVyBmhw621EbYqtozsObw1+Pi8izwWP50Xkufi6V2Kuvro1AKdt3erWh3nexK5QALYAbYot70haVa8Mfh4dX3dKXKFi+lZs3xjTRvLS6ZKsUI5zFPncxpiSUjBIi8hSEblHRKaJyBEisl0cHStJhXKcrdi+dwrlW1s+tim2MCPp44EFwERgPrC0qD0qZYWK6Vuxfe+IuCyOXProAJuTNkUXJk/6feCPwNPAfFXNUd3HsTxpEzVf8pML9cOXfkahlI4lKbr7beF7ACcBZwJ3iMj7qrpLlB0sKIrc4UJtXHJJPHfhWR50aDU1rhxp+o7C9K3ggwd3rmZ0FDrKxy6URz1wxsC8dyxu/n/R5VkX2k+YfkRxzuM63nIRZrrjAWA/YB6wX48E6ClTXM6wamvucGcK3RdqI11jOV3CM11jOepi+FEcS5nIrBedrsWRrtXx4Yf+zAWHyaOOqx51of0Uej6qc271t6NVcLqjM4oy3RFFLehCbcRVYzmKYykjUdSLjkOhuwHjulswin7EVaPbZOvudxz2rChyhwu1kStAd7S+qywPulMyiyal+RagS42dc//4H6SjyB0u1EZcNZYtD7pTrF50/Oyc+8f/IB1F7nChNuKqsWx50KElpV50KeVRJ+Wclxv/CyxFUQu6UBtx1ViO4ljKRFT1oostnUedL7tDJL561IX2U+j5OGt0m/D8v3BoylpScnbj6Gdzc/vvc0x18rNwmH4m5ZyXku7mSZeGOHKtJ06EZ59tXZ4wARYsiKb/ZSqOetHdFUdecK8fDKSpV/t9VGwbwLbrwu1DasR9a2lbClrTOlhLwjkvJ3mDtIg8D7QdZgugqnpMUXsVtSjqNBdqo22ABrc8caIF6hJX7Lzg5mZyBmhw68OMqFXJHaBx62207K/ymO6II9e6o//hdsWlpMWRFxzFPix/2V/JzpOOQhy51sYYUwRhSpVWiMghInJk8PhqHB2LVBy51sYYUwRhRtIPAZcBNwCXAhcWtUfFEEeu9YQJuV+Xb70xITU3d+95KK187nITJkgPwgXmDap6FtCvuF0qgijqNBdqY8GC9gHZsjvKQr7836jyglMpl8WRS8W2AaHS8ERonwaQZhcNvRamnvQ84CngK8CrwNmqemCubb29cGhMCYgrT9rEr7sXDs/HfTPLJcB7uLrSJsHavi935aNu24/YuT5yR7EfX9qIQ6F+RpG/bDnQyRP2ZpZxQDXwpqquirQHYW4y8aVQfqEvBkjAsURR1H38eKirg+XL3UiuuRnGjoVBg2Dhwuj2E0Ux/UJtxCFM6luh8xXmOOK4qSbMPnwo+p+UfoYRZiR9PzAB2AKcKCK/iWzvYYrg+1Iov9AXAyTgWKIo6t7c7AL0ihUuMKcD9IoVbn1zczT7iaKYfpg2fFDofDU3hzuOOIrth9mHD0X/k9LPMMLMSS9U1fEZy8+r6tG5tu30nHSYm0x8KZRf6IsBEnIsURR1zwzMaWPGtI6so9pPFEXsfbiBI4pi+74cqy/9KCQp/Uzr7pz0VhH5nogcKyJXA3UicmQkPQtzg4gvN5EU+mKAhBxLFEXdUykXkDNlBuio9lNO7HyZfMIE6T8DfYEv4eawXwbGR7L3MDeI+HITSaEvBkjIsURR1D09ks6UnvqIcj/lxM6XyadgkFbVa4GHgSXAg8AtqvqjSPYe5iYTXwrlF/pigAQcSxRF3TOnOsaMcR8kxozJnqOOYj9R3HyRlBs4Cp2vQjer+HIcpjgKZneIyGxgF2A34AfAj4GTI9l7mCL4vhTKL/TFAAk4liiKuqdSLosjcw56+fLW7I70lEd39xNFMf0wbfig0O8llQp3HHEU2w+zDx+K/ieln2GEvnAoIs+p6jEiskhVj8q1rd3MkgxR3NCQfp/Ktwzx3HwR5lgK9SOK8xFFG3H00/ipu0X/N4rIdGB7EbkAeDfS3oXhS550iejuDQ25cnqvuio7BzrXNlde2bk86UJ9DZOLXagfYdoolE8bRU54mPNVKjeiJCU/2RdhxjVfA+qAP9FaxyM+vuRJGyBcDnQUedJx9KO5OVw/O8qnjSonvNjnyydJyU/2hqrmfQBjg58pXLC+CKjMt/3YsWM1cqNGpf/esh+jRkW/LxNKc7PqtGnZv45p09z6zmzjQz/CtEENeR9RHWsc58sXhc5nOQKWaZ64mndOWkTuAepVdYqI3AbsBPwVOERVT831mqLMSadSuYcSIuFqNJqiUG0/f5rrC00LbeNDPwo9H/ZmlO4eaxznywc+3UTii67ezPKZIEB/FjgGOE9Vb8RNecTHg9xiky1MTm8ceb9R9COKfvrShilR+YbYwCPAd4A/AmcA2+GmPJ7L95qiTHfcd59qVVX258CqKrfexC7zY3n643jb5TDb+NCPpqZw/ezo43kUxxrH+fKJTXe0RwfTHR1ld5yHK1M6Q1WfCkbUnwPiTavwJU/aAOFzrbubJx1HP1KpcG10lE8bRe55FG0kSVLyk31RHt8WbiKnWjhnN0wutQ/9CNNGd/sRZh9x9MP4qbt50sZkCZPnOn58+5rTBx/cWnM6qlzZQrnDUfSj0DaFLoSFzaMuRv56Z/O1feFLbWwfdPL+L2MK57mGqTkdR65sVP3oTl81phzouPYTF19qY/vARtImcpn1PFasaJ1aaFtzuhz6kTm/PGtWa73oztbW9mU/Jn42kjZFEabmdLn0I65a0VaTujRZkDZFEabmdLn0I64caMu1Lk0WpE3kwtScLpd+ZM4Nd7W2tk/7MfGzOWnTaYXyXMPUnI4jVzaqfnSnr3HlQJdarrUvtbF9YHnSCeRDLmyYPhSqjxxHvemo9tNdcf3OfPi/YTrP8qRLiA+5sGH70DYQZn1RbY1AruChoDXhBg5R9COMQvm0YfJt46oFXSo1p00rm5NOEB9yYaPoQ3MzuQM0bn2YueI4z0WhfNqk5NuaZLKRdIL4kAsbRR8KjWTDjHR9OBfGxMHmpBPIh7rD3e1DVDWF4zgXhfpq9ZFNd3W1nrTxkA+5sD70wad+GFNMFqQTxIdc2Cj6UGjOOeycdE+fC2PiYHPSCeJDLmwUfUilACVvdkfYOem4zkWhfNqk5NuaZLI56QSKIxc2jvrIUdSbtrxgUwosT7rEFDsXNkz+caE+FGoj1/NXXdX5fO9C/UhKzWATv6T837A5aZMlivzjQm00N/uT42zKV1L+b9hI2mSJIv84TBuW42xMODYnbXKKIv+4UBs+5Dib8uXT/w3LkzadEkX+caE2LMfZmHAsSJssUeQfF2qjudlynI0Jy+akTZYo8o8LtZFK+ZPjbMpXUv5v2Jy0ySmK/OMocq2NKQc2J50gbd8zff3oH6afhXKYk1T7OCm/F1N6bLojIlEkxvtQ0D9MP8L0Myk3CoTRt2YgjdL+WProABpq4jmWUjqfpnNsJB2R7ibG+1DQP0w/wt6IkpQbBQpRJWeABrc+rt9LqZxP03k2kvaELzd42I0o2QodT6kdr/GPjaQ9khkA03oi8BXqhy/9NKYcWJD2iC83eNiNKMb4w4K0J3wpYm83omQrdDyldrzGPzYnHZHuJsb7UNA/TD/C3oiSlBsFChFxWRz5sjvi+r2Uyvk0nWc3s0QojhtA4lKoIL8v/YxLuR2viZfdzBKDmprc87adzW/24QaP8ePh4INbv2uwudktjx+fv1+lHrDK7XiNPyxIR8CXHOcoNDdDXR2sWAFjx7rlsWPdcl1duC+JNcZEx+akI1BKucOpFCxf3hqY01McY8a49WG+JNYYEx2bk45QHEXs49Lc3H5O2gK0McVhc9IxKKXc4fQUR6b01IcxJl4WpCPgS45zFDLnoMeMcSPoMWOy56iNMfGxOekI+JLjHIVUCgYNyp6DTs9RDxpkUx7GxM3mpCMUJpc2Kfm2hfKky01Sfm8mmUp/TnrePKiudsO86mq33AMK5dJGlUtdbDU1cNVV2f286ir/+hmXpPzeTGlKfpCeNw+mTIG1a91fz9q1brmHAnU+ScmlTko/42Lnw/Q4VY3sMXbsWI3dqFGq7m8l+zFqVPx9KaC5WXXatOxuTpvm1vskKf2Mi50PU2zAMs0TV5M/J51K5f+CPQ9TEZKSS52UfsbFzocpptKekx45snPre1BScqmT0s+42PkwPSn5QfqGG6CqKntdVZVb75Gk5FInpZ9xsfNhelry86TPPdf9vPpqWLfOjaBvuKF1vSeSkkudlH7Gxc6H6WnJn5NOmKTk25ZTP0spv90kU0dz0skfSSdMUuoSJ6GffWsG5v3GlIaazaHaqKlxqXTpUXJ6emPw4Ow86CScD1Oakj8nbcqSKjkDNLj1YT4gWg60SQIbSZtEKjSSDTPSLaU64KZ02UjalLXMQJ1mAdr4xIK0KWuWA218Z0HaJFKhIBp2TtpyoI3vbE7aJJKIy+LIl90Rdk7acqCN7yxP2iRaXHnSxhRTadfuMGUtivxly4E2PrMgbYwxHrMgbYwxHrMgbYwxHrMgbYwxHrMgbYwxHrMgbYwxHrMgbYwxHrMgHbO29w7ZrcfGmI5YkI5RTU12TYh07YjM4vLGGJPJgnRMrMC8MaYrrMBSTKzAvDGmK6zAUsxUIZXx+aW52QK0MeXOCix5wgrMG2M6y4J0TKzAvDGmK2xOOiZWYN4Y0xU2Jx0zKzBvjGnL5qQ9YgXmjTGdYUHaGGM8ZkHaGGM8ZkHaGGM8ZkHaGGM8Fml2h4hsBNZG1qAxxpSHUaq6U64nIg3SxhhjomXTHcYY4zEL0sYY4zEL0iZ2ItJfRB4VkUUicq9INLf0iMjCKLcL2Va1iIzPWP5XEfnXqNo3xoK06QnnA39S1aOABiDn7bAJUQ2M7+E+mBJmBZZMT3gHuEBEHlXVbwCISBVwDzAUeFVVLxWRGuALQBWwETgbUOBeYBegDjhdVT/takeCUfwcYM9gH2fh3kQODB7DgDOBvwL3AaOD/q8NHhcCg4PR9BlBsweKyHPp16rqa13tnzE2kjaxU9UngZnAIyJyu4hUAFOA11T1SGC4iBwQbP5CMOJ+DzgFGAI8DUwENgMHdbM7pwC9g32sA04K1h8CHAfUACcD2wNDVfUwYDdVvVJVZwGXA3NVdbyqbszzWmO6zIK0iZ2I7AH8FhgD7AScB+wFnBrMF+8OjAg2Xx78fAU3tfAp8GXgv4Pt+nWzO3sBXwz2eySwc7D+v4IR+gagD7AV6CsifwbmFWiz7WuN6TIL0qYnfAM4VVWbgNeASmAVcJuqjgeuwY1qAQ4Nfn4eeBM4LXjNabhph+5aBdwf7Pdy4PVg/ZY22x0KPKqqX1DVWzLWf4KbjiHjAmjb1xrTZRakTU+YBfxrMHo9FDfH/EvgBBH5A3Ax8Haw7SHBdoOBJ4E/4uaNFwM70Dri7qongF1EZBFwPfnvmF0JXCkiz4vIwyJyRLD+ZWAvEXkh6JcxkbI7Do23gguHC1V1YQ93BRE5Efgu0IgbPf9GVe/v2V6ZcmBB2hhjPGbTHcYY4zEL0sYY4zEL0sYY4zEL0sYY4zEL0sYY4zEL0sYY47H/D0HDurhOUMxVAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[:50, 0], X[:50, 1], color='red', marker='o', label='山鸢尾')\n",
    "plt.scatter(X[50:100, 0], X[50:100, 1], color='blue',\n",
    "            marker='x', label='变色鸢尾')\n",
    "plt.scatter(X[100:, 0], X[100:, 1], color='green',\n",
    "            marker='s', label='弗吉尼亚鸢尾')\n",
    "\n",
    "# 使用中文字体\n",
    "plt.rcParams['font.family'] = ['SimHei']\n",
    "\n",
    "plt.title(\"DTC基于决策数的鸢尾花分类\")  # 标题\n",
    "plt.xlabel('Sepal length')\n",
    "plt.ylabel('Sepal width')\n",
    "plt.xticks(())\n",
    "plt.yticks(())\n",
    "plt.legend(loc=2)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优缺点\n",
    "- 优点\n",
    "  - 容易解释\n",
    "  - 非参数型\n",
    "- 缺点\n",
    "  - 趋向过拟合\n",
    "  - 可能或陷于局部最小值中\n",
    "  - 没有在线学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 神经网络\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 代码实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0, loss: 0.2821310982108116\n",
      "Test_acc: 0.16666666666666666\n",
      "--------------------------\n",
      "Epoch 1, loss: 0.25459614023566246\n",
      "Test_acc: 0.16666666666666666\n",
      "--------------------------\n",
      "Epoch 2, loss: 0.22570249810814857\n",
      "Test_acc: 0.16666666666666666\n",
      "--------------------------\n",
      "Epoch 3, loss: 0.21028399094939232\n",
      "Test_acc: 0.16666666666666666\n",
      "--------------------------\n",
      "Epoch 4, loss: 0.19942265003919601\n",
      "Test_acc: 0.16666666666666666\n",
      "--------------------------\n",
      "Epoch 5, loss: 0.18873639777302742\n",
      "Test_acc: 0.5\n",
      "--------------------------\n",
      "Epoch 6, loss: 0.17851299047470093\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 7, loss: 0.16922875121235847\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 8, loss: 0.16107672825455666\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 9, loss: 0.15404684096574783\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 10, loss: 0.14802725240588188\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 11, loss: 0.14287303015589714\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 12, loss: 0.1384414080530405\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 13, loss: 0.13460607267916203\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 14, loss: 0.13126072473824024\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 15, loss: 0.12831821851432323\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 16, loss: 0.12570795230567455\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 17, loss: 0.12337299063801765\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 18, loss: 0.12126746959984303\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 19, loss: 0.11935433000326157\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 20, loss: 0.11760355532169342\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 21, loss: 0.11599067971110344\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 22, loss: 0.11449568718671799\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 23, loss: 0.11310207284986973\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 24, loss: 0.11179620958864689\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 25, loss: 0.11056671850383282\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 26, loss: 0.10940408334136009\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 27, loss: 0.10830028168857098\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 28, loss: 0.10724855586886406\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 29, loss: 0.10624313540756702\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 30, loss: 0.10527909733355045\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 31, loss: 0.10435222648084164\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 32, loss: 0.10345886461436749\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 33, loss: 0.10259588062763214\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 34, loss: 0.10176052711904049\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 35, loss: 0.10095042549073696\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 36, loss: 0.10016348026692867\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 37, loss: 0.09939785301685333\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 38, loss: 0.098651934415102\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 39, loss: 0.09792428836226463\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 40, loss: 0.09721364825963974\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 41, loss: 0.09651889279484749\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 42, loss: 0.09583901427686214\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 43, loss: 0.09517310746014118\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 44, loss: 0.09452036768198013\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 45, loss: 0.09388007409870625\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 46, loss: 0.09325156360864639\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 47, loss: 0.09263425320386887\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 48, loss: 0.09202759899199009\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 49, loss: 0.09143111668527126\n",
      "Test_acc: 0.5333333333333333\n",
      "--------------------------\n",
      "Epoch 50, loss: 0.09084436297416687\n",
      "Test_acc: 0.5666666666666667\n",
      "--------------------------\n",
      "Epoch 51, loss: 0.09026694111526012\n",
      "Test_acc: 0.5666666666666667\n",
      "--------------------------\n",
      "Epoch 52, loss: 0.08969846740365028\n",
      "Test_acc: 0.5666666666666667\n",
      "--------------------------\n",
      "Epoch 53, loss: 0.08913860842585564\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 54, loss: 0.08858705312013626\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 55, loss: 0.08804351463913918\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 56, loss: 0.08750773034989834\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 57, loss: 0.0869794450700283\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 58, loss: 0.08645843155682087\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 59, loss: 0.08594449423253536\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 60, loss: 0.08543741516768932\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 61, loss: 0.08493702299892902\n",
      "Test_acc: 0.6\n",
      "--------------------------\n",
      "Epoch 62, loss: 0.08444313704967499\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 63, loss: 0.08395560644567013\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 64, loss: 0.08347426541149616\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 65, loss: 0.0829989816993475\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 66, loss: 0.08252962119877338\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 67, loss: 0.08206604048609734\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 68, loss: 0.08160813339054585\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 69, loss: 0.08115578815340996\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 70, loss: 0.08070888184010983\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 71, loss: 0.08026731945574284\n",
      "Test_acc: 0.6333333333333333\n",
      "--------------------------\n",
      "Epoch 72, loss: 0.07983099669218063\n",
      "Test_acc: 0.6666666666666666\n",
      "--------------------------\n",
      "Epoch 73, loss: 0.07939982414245605\n",
      "Test_acc: 0.6666666666666666\n",
      "--------------------------\n",
      "Epoch 74, loss: 0.07897370308637619\n",
      "Test_acc: 0.6666666666666666\n",
      "--------------------------\n",
      "Epoch 75, loss: 0.07855255156755447\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 76, loss: 0.07813628204166889\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 77, loss: 0.07772481627762318\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 78, loss: 0.07731807045638561\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 79, loss: 0.07691597938537598\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 80, loss: 0.07651845552027225\n",
      "Test_acc: 0.7\n",
      "--------------------------\n",
      "Epoch 81, loss: 0.07612544484436512\n",
      "Test_acc: 0.7333333333333333\n",
      "--------------------------\n",
      "Epoch 82, loss: 0.075736865401268\n",
      "Test_acc: 0.7333333333333333\n",
      "--------------------------\n",
      "Epoch 83, loss: 0.07535266131162643\n",
      "Test_acc: 0.7333333333333333\n",
      "--------------------------\n",
      "Epoch 84, loss: 0.07497275248169899\n",
      "Test_acc: 0.7333333333333333\n",
      "--------------------------\n",
      "Epoch 85, loss: 0.07459708768874407\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 86, loss: 0.07422560174018145\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 87, loss: 0.07385823410004377\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 88, loss: 0.073494927957654\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 89, loss: 0.07313562650233507\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 90, loss: 0.07278027106076479\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 91, loss: 0.0724288085475564\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 92, loss: 0.07208118587732315\n",
      "Test_acc: 0.7666666666666667\n",
      "--------------------------\n",
      "Epoch 93, loss: 0.07173734437674284\n",
      "Test_acc: 0.8\n",
      "--------------------------\n",
      "Epoch 94, loss: 0.07139724120497704\n",
      "Test_acc: 0.8\n",
      "--------------------------\n",
      "Epoch 95, loss: 0.07106082793325186\n",
      "Test_acc: 0.8\n",
      "--------------------------\n",
      "Epoch 96, loss: 0.07072804402559996\n",
      "Test_acc: 0.8\n",
      "--------------------------\n",
      "Epoch 97, loss: 0.07039884477853775\n",
      "Test_acc: 0.8\n",
      "--------------------------\n",
      "Epoch 98, loss: 0.07007318455725908\n",
      "Test_acc: 0.8333333333333334\n",
      "--------------------------\n",
      "Epoch 99, loss: 0.06975101679563522\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 100, loss: 0.06943229679018259\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 101, loss: 0.06911697424948215\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 102, loss: 0.06880501098930836\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 103, loss: 0.06849636044353247\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 104, loss: 0.06819096487015486\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 105, loss: 0.06788880098611116\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 106, loss: 0.06758982595056295\n",
      "Test_acc: 0.8666666666666667\n",
      "--------------------------\n",
      "Epoch 107, loss: 0.0672939894720912\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 108, loss: 0.06700124870985746\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 109, loss: 0.06671157013624907\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 110, loss: 0.06642491556704044\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 111, loss: 0.06614124681800604\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 112, loss: 0.06586051359772682\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 113, loss: 0.06558268796652555\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 114, loss: 0.06530773546546698\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 115, loss: 0.06503561418503523\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 116, loss: 0.0647662803530693\n",
      "Test_acc: 0.9\n",
      "--------------------------\n",
      "Epoch 117, loss: 0.06449970975518227\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 118, loss: 0.06423586513847113\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 119, loss: 0.06397470366209745\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 120, loss: 0.06371620204299688\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 121, loss: 0.06346031930297613\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 122, loss: 0.06320702470839024\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 123, loss: 0.06295627728104591\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 124, loss: 0.06270805280655622\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 125, loss: 0.06246232148259878\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 126, loss: 0.062219046987593174\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 127, loss: 0.06197818834334612\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 128, loss: 0.061739739030599594\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 129, loss: 0.06150365062057972\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 130, loss: 0.061269890516996384\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 131, loss: 0.06103843729943037\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 132, loss: 0.06080926489084959\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 133, loss: 0.06058233883231878\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 134, loss: 0.060357628390192986\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 135, loss: 0.06013511214405298\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 136, loss: 0.05991475377231836\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 137, loss: 0.05969653092324734\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 138, loss: 0.05948042031377554\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 139, loss: 0.05926639400422573\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 140, loss: 0.0590544156730175\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 141, loss: 0.058844469487667084\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 142, loss: 0.05863652937114239\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 143, loss: 0.05843057204037905\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 144, loss: 0.0582265630364418\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 145, loss: 0.058024486526846886\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 146, loss: 0.057824318297207355\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 147, loss: 0.0576260294765234\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 148, loss: 0.05742960050702095\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 149, loss: 0.057235000655055046\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 150, loss: 0.05704221781343222\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 151, loss: 0.0568512286990881\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 152, loss: 0.056662000715732574\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 153, loss: 0.05647451989352703\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 154, loss: 0.05628876481205225\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 155, loss: 0.056104717776179314\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 156, loss: 0.05592234991490841\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 157, loss: 0.055741638876497746\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 158, loss: 0.05556256603449583\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 159, loss: 0.05538512021303177\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 160, loss: 0.055209274403750896\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 161, loss: 0.05503501370549202\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 162, loss: 0.05486230831593275\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 163, loss: 0.05469114799052477\n",
      "Test_acc: 0.9333333333333333\n",
      "--------------------------\n",
      "Epoch 164, loss: 0.054521515034139156\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 165, loss: 0.0543533805757761\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 166, loss: 0.05418673902750015\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 167, loss: 0.05402155965566635\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 168, loss: 0.05385783780366182\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 169, loss: 0.0536955539137125\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 170, loss: 0.05353468097746372\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 171, loss: 0.053375205025076866\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 172, loss: 0.05321711953729391\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 173, loss: 0.053060393780469894\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 174, loss: 0.05290502309799194\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 175, loss: 0.05275098606944084\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 176, loss: 0.052598257549107075\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 177, loss: 0.05244684312492609\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 178, loss: 0.0522967092692852\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 179, loss: 0.05214785411953926\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 180, loss: 0.052000248804688454\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 181, loss: 0.051853885874152184\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 182, loss: 0.05170875508338213\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 183, loss: 0.051564835011959076\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 184, loss: 0.05142211262136698\n",
      "Test_acc: 0.9666666666666667\n",
      "--------------------------\n",
      "Epoch 185, loss: 0.05128058325499296\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 186, loss: 0.051140219904482365\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 187, loss: 0.05100101511925459\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 188, loss: 0.05086294002830982\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 189, loss: 0.05072600580751896\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 190, loss: 0.05059020034968853\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 191, loss: 0.05045549105852842\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 192, loss: 0.050321875140070915\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 193, loss: 0.050189342349767685\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 194, loss: 0.050057871267199516\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 195, loss: 0.049927460961043835\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 196, loss: 0.049798086285591125\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 197, loss: 0.04966974910348654\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 198, loss: 0.04954242706298828\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 199, loss: 0.049416120164096355\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 200, loss: 0.04929080791771412\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 201, loss: 0.049166472628712654\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 202, loss: 0.04904312361031771\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 203, loss: 0.04892073292285204\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 204, loss: 0.04879929404705763\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 205, loss: 0.048678805120289326\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 206, loss: 0.048559242859482765\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 207, loss: 0.048440602608025074\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 208, loss: 0.048322876915335655\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 209, loss: 0.04820604994893074\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 210, loss: 0.04809011612087488\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 211, loss: 0.047975060530006886\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 212, loss: 0.04786088038235903\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 213, loss: 0.047747558914124966\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 214, loss: 0.047635097056627274\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 215, loss: 0.04752346966415644\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 216, loss: 0.04741268511861563\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 217, loss: 0.0473027303814888\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 218, loss: 0.047193584963679314\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 219, loss: 0.04708524979650974\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 220, loss: 0.04697770904749632\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 221, loss: 0.04687097016721964\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 222, loss: 0.046765003353357315\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 223, loss: 0.04665982071310282\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 224, loss: 0.04655539710074663\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 225, loss: 0.04645173531025648\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 226, loss: 0.04634882789105177\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 227, loss: 0.04624664783477783\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 228, loss: 0.046145220287144184\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 229, loss: 0.04604450799524784\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 230, loss: 0.04594452120363712\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 231, loss: 0.04584524501115084\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 232, loss: 0.04574666731059551\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 233, loss: 0.045648787170648575\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 234, loss: 0.04555160365998745\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 235, loss: 0.045455100014805794\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 236, loss: 0.04535927064716816\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 237, loss: 0.045264109037816525\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 238, loss: 0.04516961146146059\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 239, loss: 0.04507576674222946\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 240, loss: 0.044982570223510265\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 241, loss: 0.04489002097398043\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 242, loss: 0.0447981096804142\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 243, loss: 0.04470681771636009\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 244, loss: 0.044616153463721275\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 245, loss: 0.0445261150598526\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 246, loss: 0.0444366754963994\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 247, loss: 0.044347843155264854\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 248, loss: 0.044259609654545784\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 249, loss: 0.04417197220027447\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 250, loss: 0.04408491309732199\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 251, loss: 0.04399844445288181\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 252, loss: 0.04391254484653473\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 253, loss: 0.04382720962166786\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 254, loss: 0.043742441572248936\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 255, loss: 0.043658241629600525\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 256, loss: 0.043574594892561436\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 257, loss: 0.043491486459970474\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 258, loss: 0.04340892471373081\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 259, loss: 0.043326898477971554\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 260, loss: 0.043245404958724976\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 261, loss: 0.04316443670541048\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 262, loss: 0.04308399837464094\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 263, loss: 0.04300406854599714\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 264, loss: 0.04292464815080166\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 265, loss: 0.04284574370831251\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 266, loss: 0.04276734031736851\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 267, loss: 0.042689431458711624\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 268, loss: 0.04261201247572899\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 269, loss: 0.0425350908190012\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 270, loss: 0.042458646930754185\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 271, loss: 0.04238268081098795\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 272, loss: 0.04230719804763794\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 273, loss: 0.04223217815160751\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 274, loss: 0.04215762857347727\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 275, loss: 0.04208353627473116\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 276, loss: 0.04200990032404661\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 277, loss: 0.041936722584068775\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 278, loss: 0.041863988153636456\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 279, loss: 0.04179170075803995\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 280, loss: 0.04171985760331154\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 281, loss: 0.04164844658225775\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 282, loss: 0.0415774742141366\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 283, loss: 0.04150692466646433\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 284, loss: 0.04143681097775698\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 285, loss: 0.041367107070982456\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 286, loss: 0.04129782412201166\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 287, loss: 0.041228954680263996\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 288, loss: 0.04116049408912659\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 289, loss: 0.041092450730502605\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 290, loss: 0.04102480225265026\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 291, loss: 0.040957554243505\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 292, loss: 0.040890700183808804\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 293, loss: 0.040824237279593945\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 294, loss: 0.04075816459953785\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 295, loss: 0.04069248307496309\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 296, loss: 0.04062717594206333\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 297, loss: 0.040562246926128864\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 298, loss: 0.040497698821127415\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 299, loss: 0.040433524176478386\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 300, loss: 0.04036971554160118\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 301, loss: 0.040306273847818375\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 302, loss: 0.040243194438517094\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 303, loss: 0.040180476382374763\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 304, loss: 0.04011811036616564\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 305, loss: 0.04005609918385744\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 306, loss: 0.03999443864449859\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 307, loss: 0.039933126885443926\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 308, loss: 0.03987215645611286\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 309, loss: 0.03981153341010213\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 310, loss: 0.039751246105879545\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 311, loss: 0.03969130152836442\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 312, loss: 0.03963168291375041\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 313, loss: 0.03957238933071494\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 314, loss: 0.039513427298516035\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 315, loss: 0.03945479728281498\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 316, loss: 0.03939648298546672\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 317, loss: 0.039338490925729275\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 318, loss: 0.0392808117903769\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 319, loss: 0.03922344604507089\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 320, loss: 0.039166399743407965\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 321, loss: 0.03910965984687209\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 322, loss: 0.03905322169885039\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 323, loss: 0.03899709181860089\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 324, loss: 0.03894126508384943\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 325, loss: 0.038885734509676695\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 326, loss: 0.038830498699098825\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 327, loss: 0.03877556184306741\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 328, loss: 0.038720916491001844\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 329, loss: 0.03866655798628926\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 330, loss: 0.03861248353496194\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 331, loss: 0.03855869872495532\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 332, loss: 0.03850520262494683\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 333, loss: 0.03845197753980756\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 334, loss: 0.03839903511106968\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 335, loss: 0.03834636928513646\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 336, loss: 0.038293981458991766\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 337, loss: 0.03824185440316796\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 338, loss: 0.03819000581279397\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 339, loss: 0.038138417061418295\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 340, loss: 0.03808710025623441\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 341, loss: 0.0380360446870327\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 342, loss: 0.037985253147780895\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 343, loss: 0.037934715393930674\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 344, loss: 0.03788444120436907\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 345, loss: 0.037834417540580034\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 346, loss: 0.03778464952483773\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 347, loss: 0.03773513436317444\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 348, loss: 0.037685861345380545\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 349, loss: 0.03763684676960111\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 350, loss: 0.03758806409314275\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 351, loss: 0.03753953520208597\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 352, loss: 0.03749124752357602\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 353, loss: 0.03744319686666131\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 354, loss: 0.037395388353616\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 355, loss: 0.03734781593084335\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 356, loss: 0.03730047354474664\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 357, loss: 0.03725336538627744\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 358, loss: 0.037206490989774466\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 359, loss: 0.03715984569862485\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 360, loss: 0.0371134290471673\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 361, loss: 0.037067241966724396\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 362, loss: 0.037021270021796227\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 363, loss: 0.0369755276478827\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 364, loss: 0.03693000506609678\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 365, loss: 0.0368846976198256\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 366, loss: 0.03683961136266589\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 367, loss: 0.03679474676027894\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 368, loss: 0.03675009170547128\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 369, loss: 0.03670564712956548\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 370, loss: 0.03666141629219055\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 371, loss: 0.03661739220842719\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 372, loss: 0.03657358465716243\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 373, loss: 0.03652998013421893\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 374, loss: 0.036486583296209574\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 375, loss: 0.03644338669255376\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 376, loss: 0.036400390323251486\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 377, loss: 0.03635760163888335\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 378, loss: 0.03631501039490104\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 379, loss: 0.03627261612564325\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 380, loss: 0.036230423022061586\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 381, loss: 0.036188422702252865\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 382, loss: 0.036146617494523525\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 383, loss: 0.03610499994829297\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 384, loss: 0.0360635737888515\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 385, loss: 0.036022345535457134\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 386, loss: 0.035981299821287394\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 387, loss: 0.03594044363126159\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 388, loss: 0.03589976904913783\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 389, loss: 0.03585928771644831\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 390, loss: 0.03581898706033826\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 391, loss: 0.035778865683823824\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 392, loss: 0.03573892964050174\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 393, loss: 0.03569917008280754\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 394, loss: 0.035659585148096085\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 395, loss: 0.0356201883405447\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 396, loss: 0.03558096010237932\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 397, loss: 0.03554190881550312\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 398, loss: 0.0355030307546258\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 399, loss: 0.03546432126313448\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 400, loss: 0.035425782669335604\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 401, loss: 0.03538742661476135\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 402, loss: 0.03534922515973449\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 403, loss: 0.03531120577827096\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 404, loss: 0.03527333587408066\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 405, loss: 0.035235641058534384\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 406, loss: 0.03519811201840639\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 407, loss: 0.03516074735671282\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 408, loss: 0.035123541951179504\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 409, loss: 0.03508649580180645\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 410, loss: 0.0350496144965291\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 411, loss: 0.035012885462492704\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 412, loss: 0.03497631894424558\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 413, loss: 0.03493990423157811\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 414, loss: 0.03490366134792566\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 415, loss: 0.03486755583435297\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 416, loss: 0.0348316109739244\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 417, loss: 0.03479581791907549\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 418, loss: 0.034760172944515944\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 419, loss: 0.03472468443214893\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 420, loss: 0.03468935005366802\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 421, loss: 0.03465416096150875\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 422, loss: 0.03461911575868726\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 423, loss: 0.03458422236144543\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 424, loss: 0.03454947145655751\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 425, loss: 0.03451486863195896\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 426, loss: 0.03448040457442403\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 427, loss: 0.03444609325379133\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 428, loss: 0.03441191557794809\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 429, loss: 0.034377881325781345\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 430, loss: 0.03434399329125881\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 431, loss: 0.03431024216115475\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 432, loss: 0.03427663026377559\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 433, loss: 0.034243148285895586\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 434, loss: 0.034209813456982374\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 435, loss: 0.034176608081907034\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 436, loss: 0.03414353961125016\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 437, loss: 0.03411060944199562\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 438, loss: 0.03407781198620796\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 439, loss: 0.03404514258727431\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 440, loss: 0.03401260683313012\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 441, loss: 0.03398020751774311\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 442, loss: 0.033947933465242386\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 443, loss: 0.03391579119488597\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 444, loss: 0.03388377698138356\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 445, loss: 0.03385188803076744\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 446, loss: 0.033820129465311766\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 447, loss: 0.033788494765758514\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 448, loss: 0.03375698812305927\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 449, loss: 0.03372560814023018\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 450, loss: 0.03369434829801321\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 451, loss: 0.033663210459053516\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 452, loss: 0.03363219555467367\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 453, loss: 0.03360130824148655\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 454, loss: 0.0335705429315567\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 455, loss: 0.033539886586368084\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 456, loss: 0.033509363885968924\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 457, loss: 0.03347895108163357\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 458, loss: 0.03344866121187806\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 459, loss: 0.03341847192496061\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 460, loss: 0.03338842187076807\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 461, loss: 0.03335847798734903\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 462, loss: 0.03332865051925182\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 463, loss: 0.033298938535153866\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 464, loss: 0.03326933737844229\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 465, loss: 0.0332398503087461\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 466, loss: 0.03321047965437174\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 467, loss: 0.03318122075870633\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 468, loss: 0.033152070827782154\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 469, loss: 0.03312302753329277\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 470, loss: 0.033094103913754225\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 471, loss: 0.03306528413668275\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 472, loss: 0.03303657611832023\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 473, loss: 0.033007973339408636\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 474, loss: 0.03297947905957699\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 475, loss: 0.032951085828244686\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 476, loss: 0.03292280342429876\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 477, loss: 0.03289463138207793\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 478, loss: 0.032866557128727436\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 479, loss: 0.03283859323710203\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 480, loss: 0.032810724806040525\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 481, loss: 0.032782965805381536\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 482, loss: 0.03275530878454447\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 483, loss: 0.03272774862125516\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 484, loss: 0.032700291369110346\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 485, loss: 0.03267294308170676\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 486, loss: 0.03264568466693163\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 487, loss: 0.032618530094623566\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 488, loss: 0.03259148122742772\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 489, loss: 0.032564531080424786\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 490, loss: 0.03253766708076\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 491, loss: 0.03251091064885259\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 492, loss: 0.032484248746186495\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 493, loss: 0.032457681372761726\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 494, loss: 0.03243120992556214\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 495, loss: 0.03240483067929745\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 496, loss: 0.03237855341285467\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 497, loss: 0.03235236555337906\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 498, loss: 0.032326264306902885\n",
      "Test_acc: 1.0\n",
      "--------------------------\n",
      "Epoch 499, loss: 0.03230027295649052\n",
      "Test_acc: 1.0\n",
      "--------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm1ElEQVR4nO3deXxddZ3/8dcnd8vaJE3SLelCSwFbllZr2SoiMlocf4ArRZ1BB+WHIzr+REd86M+FceYnOm6MOD9QERcWFRWLosCw6lC0BbpDoY2lTUpJmr3Zl8/8cU/CbXrbJm1uT5L7fj4e95Fzvuec5HNKuO98v99zzjV3R0REZLicsAsQEZHxSQEhIiJpKSBERCQtBYSIiKSlgBARkbQUECIikpYCQiQEZrbfzOaHXYfI4SggJDRmttPMLgzh595mZj3Bm/Tg67IM/rxHzeyDqW3uXuju1Rn6ee8xs3XBeb1kZr83sxWZ+FkyuSkgJFt9NXiTHnz9LOyCxoKZfQL4FvBvwHRgDvBd4JKj+F7RMS1OJhwFhIw7ZpYws2+Z2Z7g9S0zSwTbys3st2bWbGaNZvZHM8sJtn3azGrNrM3MtpnZG0f5c28zsy+nrJ9vZjUp6zvN7JNmttHMWszsZ2aWm7L9EjNbb2atZrbDzFaa2b8CrwO+E/xF/51gXzezE4PlYjP7sZnVm9mLZva5lHN6v5n9ycz+3cyazOyvZnbRIeovBq4HPuLuv3L3dnfvdfd73f1TozjHT5vZRqA9WL572M/5tpndmFL7D4KeSq2ZfdnMIqP5d5fxS38hyHj0WeAsYAngwG+AzwH/F7gWqAEqgn3PAtzMTgauAV7r7nvMbB6QiTeqdwMrgS7gv4H3A//fzJYDPwbeCTwEzASK3P0PZnYu8FN3//4hvud/AMXAfKAMeAB4CfhBsP1M4EdAOXAV8AMzq/SDn5NzNpAL/PoYz/Fy4G+BfcA04AtmVuTubcGb/7uBtwX73gbUAScCBcBvgd3AzcdYg4wD6kHIePRe4Hp3r3P3euBLwN8F23pJvvnODf46/mPwRtkPJIBFZhZz953uvuMwP+OTQS+k2cz2jaK2G919j7s3AveSDDGAK4Fb3f1Bdx9w91p3f+5I3yx4w10FfMbd29x9J/D1lPMFeNHdv+fu/SSDYibJ4aPhyoB97t43ivNJ50Z33+3une7+IvA0rwTCBUCHuz9pZtOBtwAfD3ordcA3g/ORSUABIePRLODFlPUXgzaArwHbgQfMrNrMrgNw9+3Ax4EvAnVmdpeZzeLQ/t3dS4JX+Shq25uy3AEUBsuzgcMF0qGUAzEOPt/KdD/T3TuCxUIO1gCUj8Hcwe5h63eQ7FUAvCdYB5hLsvaXBsOWZM9h2jH+fBknFBAyHu0h+eYzaE7QRvBX9rXuPh+4GPjE4FyDu9/h7iuCYx24YZQ/tx3IT1mfMYpjdwMLDrHtcI9M3keyVzT8fGtH8bMHrQG6gUsPs89IznF4vb8AzjezKpI9icGA2B38vPKUsJ3i7ouPonYZhxQQEraYmeWmvKLAncDnzKzCzMqBzwM/BTCzt5rZiWZmQAvJoaUBMzvZzC4IJrO7gE5gYJS1rAfeYmZTzWwGyR7JSP0A+ICZvdHMcsys0sxOCba9THJ+4SDBsNHPgX81syIzmwt8YvB8R8PdW0j+W91kZpeaWb6ZxczsIjP76tGeYzDM9yjwQ+Cv7v5s0P4SyfmSr5vZlOC8F5jZ60dbu4xPCggJ230k38wHX18EvgysAzYCm0iOgQ9eebMQ+C9gP8m/mL/r7o+QnH/4Csm/yPeSHOb4zChr+QmwAdhJ8o1vxJe+uvtfgA+QHINvAR7jlV7Bt4F3Blch3Zjm8I+S/Mu+GvgTyb/Qbx1l7YN1fJ1kwHwOqCf5V/41wD3BLkd7jncAF/JK72HQ3wNxYCvQBNxNco5EJgHTBwaJiEg66kGIiEhaCggREUlLASEiImkpIEREJK1J86iN8vJynzdvXthliIhMKE899dQ+d69It23SBMS8efNYt25d2GWIiEwoZvbiobZpiElERNJSQIiISFoKCBERSWvSzEGIiIxWb28vNTU1dHV1hV1KxuXm5lJVVUUsFhvxMQoIEclaNTU1FBUVMW/ePJLPf5yc3J2GhgZqamo44YQTRnychphEJGt1dXVRVlY2qcMBwMwoKysbdU9JASEiWW2yh8OgoznPrA+I9u4+vvHg8zyzqynsUkRExpWsD4juvgFufOgFNuxuDrsUEZFxJesDIh5N/hP09I/2w8dERCY3BUQkCIg+BYSIhOPmm2/mIx/5SNhlHCTrAyIWSU7cKCBEJCybNm3itNNOC7uMg2R9QJgZ8WgO3RpiEpGQbNy48aCAeO6557jgggtYsmQJF154Ifv27QPgRz/6Ea95zWs4/fTTWbFixSHbxoJulAMSkRz1IESy3Jfu3cLWPa1j+j0XzZrCF/7X4iPut3nzZk499dSh9e7ubt7xjndw++23s2TJEm644Qa++c1vct1113HDDTewfv164vE4zc3NtLW1HdQ2VrK+BwHJiWoFhIiEYffu3RQVFVFcXDzUds8997BixQqWLFkCwKJFi6irqyMSidDZ2cm1117LunXrKCkpSds2VtSDQAEhIozoL/1MSDf/sHXr1gPaNm3axKJFi8jPz2fz5s3ce++9XHXVVXzwgx/kH//xH9O2jQUFBEFAaA5CREKQbv6hsrKS9evXA1BdXc1PfvIT/vSnP/HCCy+wcOFCVq1axdatW+nq6krbNlYUECQvdVUPQkTCsGnTJv7whz9w5513AjBz5kwefvhh7rvvPk477TTy8vK49dZbKSsr49prr2XNmjUUFBSwePFivve973H11Vcf1DZWFBBoiElEwnP77benbb/nnnsOarvttttG1DZWNEmNhphERNJRQJAcYupWD0JE5AAKCDTEJJLN3D3sEo6LozlPBQSQUECIZKXc3FwaGhomfUgMfqJcbm7uqI7TJDWagxDJVlVVVdTU1FBfXx92KRk3+JnUo6GAQJe5imSrWCw2qs9ozjYaYkJzECIi6Sgg0BCTiEg6CgggHomoByEiMowCAg0xiYiko4AA4hGjp39g0l/qJiIyGgoIkj0IQPMQIiIpMhoQZrbSzLaZ2XYzuy7N9k+Y2VYz22hmD5nZ3JRt/Wa2PnitzmSdQwGhYSYRkSEZuw/CzCLATcDfADXAWjNb7e5bU3Z7Bljm7h1m9mHgq8BlwbZOd1+SqfpSxSMKCBGR4TLZg1gObHf3anfvAe4CLkndwd0fcfeOYPVJYHS3+Y2ReDQCaIhJRCRVJgOiEtidsl4TtB3KlcDvU9ZzzWydmT1pZpemO8DMrgr2WXcst8priElE5GDj4lEbZvY+YBnw+pTmue5ea2bzgYfNbJO770g9zt1vAW4BWLZs2VFfgqSAEBE5WCZ7ELXA7JT1qqDtAGZ2IfBZ4GJ37x5sd/fa4Gs18CiwNFOFDs5B6DMhRERekcmAWAssNLMTzCwOrAIOuBrJzJYCN5MMh7qU9lIzSwTL5cC5QOrk9phK6DJXEZGDZGyIyd37zOwa4H4gAtzq7lvM7HpgnbuvBr4GFAK/MDOAXe5+MfAq4GYzGyAZYl8ZdvXTmNIQk4jIwTI6B+Hu9wH3DWv7fMryhYc47gngtEzWlkoBISJyMN1Jje6DEBFJRwGBHrUhIpKOAgINMYmIpKOAQENMIiLpKCB45TLXbg0xiYgMUUCgISYRkXQUECggRETSUUCgOQgRkXQUEEA0kkOOQU9/f9iliIiMGwqIQDyaox6EiEgKBUQgHsmht/+onxguIjLpKCAC8WhEj/sWEUmhgAgkojl092kOQkRkkAIikBeP0NWrgBARGaSACBTEI+zvVkCIiAxSQAQKElHau/vCLkNEZNxQQAQUECIiB1JABAoTUdp7FBAiIoMUEIGCRIR2zUGIiAxRQAQKElH2a4hJRGSIAiJQEI/S0zdArz4TQkQEUEAMKUhEAejQMJOICKCAGFKYiACwXxPVIiKAAmLIYA9Cl7qKiCQpIAKDAaGJahGRJAVEYEpuMiBaO3tDrkREZHxQQARK8uMAtCggREQABcSQkrwYAE3tPSFXIiIyPiggAsVBQDSrByEiAigghkQjOUzJjdLcoYAQEYEMB4SZrTSzbWa23cyuS7P9E2a21cw2mtlDZjY3ZdsVZvZC8Loik3UOKsmP09yhISYREchgQJhZBLgJuAhYBFxuZouG7fYMsMzdTwfuBr4aHDsV+AJwJrAc+IKZlWaq1kGl+TGa1IMQEQEy24NYDmx392p37wHuAi5J3cHdH3H3jmD1SaAqWH4z8KC7N7p7E/AgsDKDtQJQrB6EiMiQTAZEJbA7Zb0maDuUK4Hfj+ZYM7vKzNaZ2br6+vpjLDfZg9AktYhI0riYpDaz9wHLgK+N5jh3v8Xdl7n7soqKimOuozQ/rstcRUQCmQyIWmB2ynpV0HYAM7sQ+Cxwsbt3j+bYsVacF6O1q48+PfJbRCSjAbEWWGhmJ5hZHFgFrE7dwcyWAjeTDIe6lE33A28ys9JgcvpNQVtGleYn74Vo7dLzmEREopn6xu7eZ2bXkHxjjwC3uvsWM7seWOfuq0kOKRUCvzAzgF3ufrG7N5rZv5AMGYDr3b0xU7UOKi1IPm6jqaOHqcGyiEi2ylhAALj7fcB9w9o+n7J84WGOvRW4NXPVHWzobmpd6ioiMj4mqceL0uCBfbrUVUREAXGAwYDQzXIiIgqIAxTnDw4xqQchIqKASDElN0okxzQHISKCAuIAZkZJXowm9SBERBQQwxXrcRsiIoAC4iClemCfiAiggDhIaX5McxAiIiggDlKcF1dAiIiggDhI8kODNMQkIqKAGKYkP0ZHTz/dff1hlyIiEioFxDBDD+xr1zCTiGQ3BcQw5YUJAPbt7z7CniIik5sCYpjywmQPQgEhItlOATHMYA+iYb8mqkUkuykghinTEJOICKCAOEhBPEJuLEcBISJZTwExjJlRXpjQEJOIZD0FRBplhQnq1YMQkSyngEijojCuHoSIZL0RBYSZFZhZTrB8kpldbGaxzJYWnrKChOYgRCTrjbQH8TiQa2aVwAPA3wG3ZaqosJUXxWlo72FgwMMuRUQkNCMNCHP3DuDtwHfd/V3A4syVFa6yggT9A06LPjhIRLLYiAPCzM4G3gv8LmiLZKak8JUX6V4IEZGRBsTHgc8Av3b3LWY2H3gkY1WFbPBxG7qSSUSyWXQkO7n7Y8BjAMFk9T53/1gmCwvTtKJcAOpaFRAikr1GehXTHWY2xcwKgM3AVjP7VGZLC09lSR4Atc2dIVciIhKekQ4xLXL3VuBS4PfACSSvZJqU8uIRphbEFRAiktVGGhCx4L6HS4HV7t4LTOprQGeV5LJHASEiWWykAXEzsBMoAB43s7lAa6aKGg9mFedR26SAEJHsNaKAcPcb3b3S3d/iSS8Cb8hwbaGqLM1jT3Mn7pO6oyQickgjnaQuNrNvmNm64PV1kr2JIx230sy2mdl2M7suzfbzzOxpM+szs3cO29ZvZuuD1+oRn9EYqSzJo72nn9bOvuP9o0VExoWRDjHdCrQB7w5ercAPD3eAmUWAm4CLgEXA5Wa2aNhuu4D3A3ek+Rad7r4keF08wjrHzCxdySQiWW5E90EAC9z9HSnrXzKz9Uc4Zjmw3d2rAczsLuASYOvgDu6+M9g2MNKCj5fUgFg0a0rI1YiIHH8j7UF0mtmKwRUzOxc40p/WlcDulPWaoG2kcoPhrCfN7NJ0O5jZVYPDXvX19aP41kc2eC+ErmQSkWw10h7E1cCPzaw4WG8CrshMSUPmuntt8FiPh81sk7vvSN3B3W8BbgFYtmzZmM4mlxXESURz2NXYMZbfVkRkwhjpVUwb3P0M4HTgdHdfClxwhMNqgdkp61VB24i4e23wtRp4FFg60mPHQk6OMb+ikO11+4/njxURGTdG9Yly7t4a3FEN8Ikj7L4WWGhmJ5hZHFgFjOhqJDMrNbNEsFwOnEvK3MXxcuI0BYSIZK9j+chRO9xGd+8DrgHuB54Ffh48CfZ6M7sYwMxea2Y1wLuAm81sS3D4q4B1ZraB5FNjv+Luxz0gFk4rpLa5k44eXeoqItlnpHMQ6RxxzN/d7wPuG9b2+ZTltSSHnoYf9wRw2jHUNiZOnFYIQHV9O6dWFh9hbxGRyeWwAWFmbaQPAgPyMlLRODIYENvr9isgRCTrHDYg3L3oeBUyHs0rKyCSY5qHEJGsdCxzEJNePJrD3Kn5vFDXFnYpIiLHnQLiCF41cwqbayf1g2tFRNJSQBzB0jkl1DZ3UtfWFXYpIiLHlQLiCJbMLgFg/a7mUOsQETneFBBHcGplMdEcY0NNc9iliIgcVwqII8iNRThlZhHrdzeHXYqIyHGlgBiBJbNL2LC7hb7+cfdUchGRjFFAjMA5C8rZ392nXoSIZBUFxAicu6CcHIPHnh/bz5wQERnPFBAjUJwfY8nsEh5XQIhIFlFAjNB5J1WwsbaFxvaesEsRETkuFBAj9MZTpuMOD2zZG3YpIiLHhQJihE6tnMK8snxWb9gTdikiIseFAmKEzIyLz5jFmuoG6lr12A0RmfwUEKNw8ZJZuMM960f80doiIhOWAmIUTpxWxPJ5U/npk7voHzjiB+qJiExoCohR+ruz57KrsYPHnq8LuxQRkYxSQIzSylNnMH1Kglserw67FBGRjFJAjFIsksOHXjefJ6sb+ctfG8MuR0QkYxQQR+G9Z86lvDDBNx7chrvmIkRkclJAHIW8eISPXnAiT1Y3cr9unBORSUoBcZTee+YcTplRxL/89lm6evvDLkdEZMwpII5SNJLDFy9eTG1zJ999ZHvY5YiIjDkFxDE4a34Zb1tayXcf3cFGfSSpiEwyCohj9MWLF1NRlODjP1tPZ4+GmkRk8lBAHKPivBhff9cZVNe38/nfbNZVTSIyaSggxsA5J5bzsQtO5BdP1fDTP+8KuxwRkTGhgBgjH7/wJN5wcgVfWr2FJ3bsC7scEZFjltGAMLOVZrbNzLab2XVptp9nZk+bWZ+ZvXPYtivM7IXgdUUm6xwLOTnGt1YtZV55Af/7x0+xdU9r2CWJiByTjAWEmUWAm4CLgEXA5Wa2aNhuu4D3A3cMO3Yq8AXgTGA58AUzK81UrWOlOC/Gj/9hOYW5Ua744V/Y3dgRdkkiIkctkz2I5cB2d6929x7gLuCS1B3cfae7bwQGhh37ZuBBd2909ybgQWBlBmsdM7NK8vjxPyynp2+A93z/SYWEiExYmQyISmB3ynpN0DZmx5rZVWa2zszW1dfXH3WhY23h9CJ+cuVyWjv7uOzmNfx1X3vYJYmIjNqEnqR291vcfZm7L6uoqAi7nAOcXlXCHR86k66+AS67eQ0vvNwWdkkiIqOSyYCoBWanrFcFbZk+dtxYPKuYu646iwGHd9+8hnU79XhwEZk4MhkQa4GFZnaCmcWBVcDqER57P/AmMysNJqffFLRNOCdNL+Luq8+mJD/Oe77/Z+7dsCfskkRERiRjAeHufcA1JN/YnwV+7u5bzOx6M7sYwMxea2Y1wLuAm81sS3BsI/AvJENmLXB90DYhzSsv4FcfPoczqor56J3PcNMj23XHtYiMezZZ3qiWLVvm69atC7uMw+rq7efTv9zIb9bvYeXiGXztXadTlBsLuywRyWJm9pS7L0u3bUJPUk80ubEI37psCZ/721fx4LMvc8l3/luT1yIybikgjjMz44Ovm88dHzyT1q4+Lrnpv/n1MzVhlyUichAFREjOnF/G7z62gsWzpvB/fraBj935DC2dvWGXJSIyRAERoulTcrnzQ2fxyTedxH2bXuKibz3Omh0NYZclIgIoIEIXjeRwzQUL+eWHzyERi/Ce7z/Jv933rD58SERCp4AYJ86YXcJvP7qCVa+dwy2PV7Py24/zxHY9NlxEwqOAGEcKElH+39tP444PnYkB7/n+n/nnuzfQ0qG5CRE5/hQQ49A5C8r5w8fP4+rXL+CXT9fyxm88xi/W7WZgYHLcsyIiE4MCYpzKjUW47qJT+M1HzmXO1Dw+dfdG3vafT/DMrqawSxORLKGAGOdOrSzm7qvP4RvvPoOXmjt523ef4Nqfb6CutSvs0kRkklNATAA5OcbbX13Fw588n6tfv4B7N+zh/H9/lG88+DxtXZqfEJHM0LOYJqCd+9r52gPb+N3GlyjNj/GRN5zI+86aS24sEnZpIjLBHO5ZTAqICWxTTQtfvf85/vjCPmYW5/KxNy7kHa+uIh5Vx1BERkYBMck9sX0fN9y/jQ27m5lVnMtV581n1fI56lGIyBEpILKAu/PY8/Xc9Mh21u5sorwwwQdfdwLvO2suhYlo2OWJyDilgMgyf65u4DuPbOePL+yjOC/Ge8+cw9+fPY8ZxblhlyYi44wCIkut393Mfz66nQe3vkyOGW85bSYfOHceS+eUhl2aiIwTCogst7uxgx89sZOfrd1NW3cfS+eU8P5z5vHmxTM0TyGS5RQQAsD+7j7uXreb257Yyc6GDkryY7x9aRWXL5/NwulFYZcnIiFQQMgBBgacJ3Y0cOfaXTywZS+9/c5r5pay6rWzeevps8iLq1chki0UEHJIDfu7+dXTtdy5dhfV9e0UxCO8+dQZXLqkknMWlBGN6J4KkclMASFH5O6s3dnEr56u4XebXqKtq4/ywgRvPX0mly6t5IyqYsws7DJFZIwpIGRUunr7eXRbHfc8s4eHn6ujp3+AuWX5rDx1Bm9ePIMlVSXk5CgsRCYDBYQctZbOXu7fvJd7N+5hzY4G+gac6VMSvHlxMiyWnzCVmIahRCYsBYSMiZaOXh7e9jJ/2LyXx56vp6t3gJL8GG84eRrnn1zBeQsrKC2Ih12miIyCAkLGXGdPP489X8/9W/by6LY6mjp6MYMzqko4/+QKzj95GqdXFmsoSmScU0BIRvUPOBtrmnns+Xoe3VbPhppm3GFqQZzXLSznnAVlnD2/nNlT8zTRLTLOKCDkuGps7+GPL9Tz2LZ6Hn9hH/v2dwNQWZLHmfOncvb8Ms5eUEZVaX7IlYqIAkJC4+7sqN/Pmh0NrKlu4MnqRhrbewCYPTWP186byqvnlPKauaWcNL2IiIakRI4rBYSMGwMDzvN1bazZ0cCT1Q089WLzUA+jMBFl6ZySocBYMqeEKbmxkCsWmdwUEDJuuTu7Gzt5alcjT73YxFMvNrNtbysDDmZwQnkBp1UWD70WVxbr8y1ExtDhAiKj/6eZ2Urg20AE+L67f2XY9gTwY+A1QANwmbvvNLN5wLPAtmDXJ9396kzWKuEwM+aU5TOnLJ+3La0CoK2rlw27W3h6VxMba1r4c3Ujv1m/J9j/4NA4ZeYUivPU0xAZaxkLCDOLADcBfwPUAGvNbLW7b03Z7Uqgyd1PNLNVwA3AZcG2He6+JFP1yfhVlBtjxcJyViwsH2qrb+tmc20Lm2pbDgoNgFnFuZw8o4iTZ0zhlBlFnDyjiAUVhfp8bpFjkMkexHJgu7tXA5jZXcAlQGpAXAJ8MVi+G/iO6TpISaOiKMEbTpnGG06ZNtQ2GBrP7W1j295Wntvbxp+276O3PzlsGs0x5lcUcPKMKZw8vZAFFYUsmFbI3LJ8ElE9sVbkSDIZEJXA7pT1GuDMQ+3j7n1m1gKUBdtOMLNngFbgc+7+x+E/wMyuAq4CmDNnzthWL+NeutDo7R/gr/vah0Jj2942ntnVxL0bXult5BjMnprP/PIC5lckg2N+RQELKgopL4zrXg2RwHid7XsJmOPuDWb2GuAeM1vs7q2pO7n7LcAtkJykDqFOGWdikRxOml7ESdOL4IxZQ+3t3X38dV87O+r3s6M++bW6vp011Q109Q4M7VeUG2VeWUFyXmRqPnOnJr/OKctnZnGeLsOVrJLJgKgFZqesVwVt6fapMbMoUAw0ePLSqm4Ad3/KzHYAJwG6TEmOSkEiyqmVxZxaWXxA+8CAs6elk+qU0HixsYMttS3cv3kvfQOv/N0RixhVpfnMTgmO2VPzqSrNY1ZJHqX5MfU+ZFLJZECsBRaa2Qkkg2AV8J5h+6wGrgDWAO8EHnZ3N7MKoNHd+81sPrAQqM5grZKlcnKSb/pVpfmcd1LFAdv6+gd4qaWL3Y0dvNjYwa7GDnY1JL+u39VEa1ffAfvnxnKYVZJHZUkeM4tzmVWSN7Q+K2jTZ4DLRJKxgAjmFK4B7id5meut7r7FzK4H1rn7auAHwE/MbDvQSDJEAM4DrjezXmAAuNrdGzNVq0g60UgOs4Newjlptrd09LKrsYM9LZ3saR58dVHb3Mm2vfXUtXUfdEx5YZyZxXlMn5Jg2pRcphflMm1KIrlelMv0KbmUFcT1kEMZF3SjnEiGdPf183JLN7XNKQHSkgyRurZu6lq7aAgeO5IqmmNUFCWYVhSEyJQE04PwKC+KU16YoKwwQVlBXD0SOWah3Sgnks0S0cjQTYCH0tM3QP3+ZFi83NpNXVsXLw8td7O7sYN1Oxtp6uhNe3xhIkpZYZyygjhlhQnKCxOUp6yXFcaDtgQleTH1TGRUFBAiIYpHc6gM5ikOp6u3n/q2bvbt76Zhfw8N7d3s299Dw/6eZFt7Mkye2dVMY3s3A2kGBnIMSvLjlOTHKM2PU5ofoyTl62D7K9uTy+qlZC8FhMgEkBuLDM2HHEn/gNPc0UNDe88rgbI/GShNHT00d/TS1NFDbXMXW/a00tTRc8ClvsPlxSKvhElBECZ5MabkxZiSG2NKXjT4GmNKbvSAdt2QOLEpIEQmmUiOBcNLieT9ICPQ1ds/FBypIdLc0UtTew9NHb00B9team6ltauXls7eobvWDyURzRkKjuJDhkpyvTARvHKjFMSjFOVGKUhE9ZnnIVJAiAi5sQgziiPMKM4d8THuTnffAC2dvbR29tLa1UtrZ1/wtZfWrr6D2pvae3ixoYPWzmTA9KUbCxsmEc05IDgKc5NBUjAYKIkIhYkYBYnIUKgMD5uCRJT8eIRENEf3qoyCAkJEjoqZkRuLkBuLMH3KyINlkLvT1Tsw1Btp6+qjvbuP/YOv4evdyfW2rj7q2rrYX9/H/u5+9nf3HnaILFWOQX48GRb58Qh58SgF8Qh5wXpBPDq0nLrf4HJePEJBIkpeLNg/EewfixCdhD0dBYSIhMLMyAvedI8mYFL19Q/Q3t3P/p5ksKQGyv6uPtp7+ujo6aezp5/2nj46e/rp6OmnI2hv6+qjrrX7gG2dvf2jqiEezUmGSBCayVfO0HpeLEIiljO0nLotkaZt8PhX9g++RzTnuF2NpoAQkQkvGsmhOD+H4vyx+1yQgQGns7f/gGA5XMgMLnf19tPZO0BXb//Qq62rj86U9a7eATp7++kfwRBbOolozgEBdFpVCf9x+dIxO/dBCggRkTRycoyCYK4jU3r7B14Jjp4BuvqSy509/XT1DdDZ0093X7CeJngGg6aq9PCXSR8tBYSISEhikRxikZxx+9nrk29WRURExoQCQkRE0lJAiIhIWgoIERFJSwEhIiJpKSBERCQtBYSIiKSlgBARkbQmzUeOmlk98OIxfItyYN8YlTNR6Jyzg845OxztOc9194p0GyZNQBwrM1t3qM9lnax0ztlB55wdMnHOGmISEZG0FBAiIpKWAuIVt4RdQAh0ztlB55wdxvycNQchIiJpqQchIiJpKSBERCStrA8IM1tpZtvMbLuZXRd2PWPFzG41szoz25zSNtXMHjSzF4KvpUG7mdmNwb/BRjN7dXiVHz0zm21mj5jZVjPbYmb/FLRP2vM2s1wz+4uZbQjO+UtB+wlm9ufg3H5mZvGgPRGsbw+2zwv1BI6BmUXM7Bkz+22wPqnP2cx2mtkmM1tvZuuCtoz+bmd1QJhZBLgJuAhYBFxuZovCrWrM3AasHNZ2HfCQuy8EHgrWIXn+C4PXVcB/Hqcax1ofcK27LwLOAj4S/PeczOfdDVzg7mcAS4CVZnYWcAPwTXc/EWgCrgz2vxJoCtq/Gew3Uf0T8GzKejac8xvcfUnK/Q6Z/d1296x9AWcD96esfwb4TNh1jeH5zQM2p6xvA2YGyzOBbcHyzcDl6fabyC/gN8DfZMt5A/nA08CZJO+ojQbtQ7/nwP3A2cFyNNjPwq79KM61KnhDvAD4LWBZcM47gfJhbRn93c7qHgRQCexOWa8J2iar6e7+UrC8F5geLE+6f4dgGGEp8Gcm+XkHQy3rgTrgQWAH0OzufcEuqec1dM7B9hag7LgWPDa+BfwzMBCslzH5z9mBB8zsKTO7KmjL6O929GgrlYnN3d3MJuU1zmZWCPwS+Li7t5rZ0LbJeN7u3g8sMbMS4NfAKeFWlFlm9lagzt2fMrPzQy7neFrh7rVmNg140MyeS92Yid/tbO9B1AKzU9argrbJ6mUzmwkQfK0L2ifNv4OZxUiGw+3u/qugedKfN4C7NwOPkBxeKTGzwT8AU89r6JyD7cVAw/Gt9JidC1xsZjuBu0gOM32byX3OuHtt8LWO5B8Cy8nw73a2B8RaYGFw9UMcWAWsDrmmTFoNXBEsX0FyjH6w/e+DKx/OAlpSuq0ThiW7Cj8AnnX3b6RsmrTnbWYVQc8BM8sjOefyLMmgeGew2/BzHvy3eCfwsAeD1BOFu3/G3avcfR7J/2cfdvf3MonP2cwKzKxocBl4E7CZTP9uhz3xEvYLeAvwPMlx28+GXc8YntedwEtAL8nxxytJjrs+BLwA/BcwNdjXSF7NtQPYBCwLu/6jPOcVJMdpNwLrg9dbJvN5A6cDzwTnvBn4fNA+H/gLsB34BZAI2nOD9e3B9vlhn8Mxnv/5wG8n+zkH57YheG0ZfK/K9O+2HrUhIiJpZfsQk4iIHIICQkRE0lJAiIhIWgoIERFJSwEhIiJpKSBERsHM+oOnaQ6+xuwJwGY2z1KevisSNj1qQ2R0Ot19SdhFiBwP6kGIjIHgWf1fDZ7X/xczOzFon2dmDwfP5H/IzOYE7dPN7NfB5zhsMLNzgm8VMbPvBZ/t8EBwd7RIKBQQIqOTN2yI6bKUbS3ufhrwHZJPGwX4D+BH7n46cDtwY9B+I/CYJz/H4dUk746F5PP7b3L3xUAz8I6Mno3IYehOapFRMLP97l6Ypn0nyQ/uqQ4eGLjX3cvMbB/J5/D3Bu0vuXu5mdUDVe7enfI95gEPevLDXzCzTwMxd//ycTg1kYOoByEydvwQy6PRnbLcj+YJJUQKCJGxc1nK1zXB8hMknzgK8F7gj8HyQ8CHYegDf4qPV5EiI6W/TkRGJy/49LZBf3D3wUtdS81sI8lewOVB20eBH5rZp4B64ANB+z8Bt5jZlSR7Ch8m+fRdkXFDcxAiYyCYg1jm7vvCrkVkrGiISURE0lIPQkRE0lIPQkRE0lJAiIhIWgoIERFJSwEhIiJpKSBERCSt/wHq+4EO5f+PiwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeE0lEQVR4nO3de5RcZZ3u8e+TG50b6dxoIJ2kA4TJxQRkmoCCioIIikFmVIjMEhzGzHgAcfBwQM0Chxk9jrrGo0dwhJmR4SzuooiIAQ0oKtdASEgTMCEm0iGQkO4kpDud6nT/zh+1q1J0Okkn6eqqrv181qqV2rt21f7tpumn3v2+796KCMzMLL0GlLoAMzMrLQeBmVnKOQjMzFLOQWBmlnIOAjOzlHMQmJmlnIPAzCzlHARW0ST9RlKzpEOK8NmS9HlJyyW1SGqUdI+kWb29L7NichBYxZJUB7wHCGBuEXbxXeAK4PPAGOBY4D7gI/v7QZIG9WplZvvBQWCV7NPAk8AtwEWFL0iaKOknkjZK2iTp+wWvfVbSCklvSXpR0gldP1jSVOBSYF5EPBIROyKiNSJui4hvJNv8RtLfFbznYkm/L1gOSZdKWgmslPQDSd/usp+fSboyeX6kpHuTmv8k6fO98DMycxBYRfs0cFvy+JCkGgBJA4EHgLVAHTABuDN57RPAV5P3Hkq2JbGpm88+HWiMiKcPssaPAScBM4A7gPMlKallNHAmcKekAcDPgaVJvacDX5D0oYPcv5mDwCqTpFOBycDdEfEs8ArwqeTlOcCRwFUR0RIRbRGR+6b+d8A3I+KZyFoVEWu72cVYYH0vlPq/I6IpIrYDvyN7Gus9yWsfB56IiNeAE4HxEXF9RGQiYjVwM3BBL9RgKecgsEp1EfBwRLyZLN/OrtNDE4G1EbGzm/dNJBsa+7IJOOKgq4RXc08iewXIO4F5yapPkW3NQDbUjpS0OfcAvgzU9EINlnLuoLKKI2ko8ElgoKTXk9WHANWSjiP7x3eSpEHdhMGrwNE92M0i4AZJ9RGxeA/btADDCpYP72abrpf/vQN4WNI3yJ4yOq+grj9FxNQe1Ga2X9wisEr0MaCD7Hn345PHdLKnXj4NPE32tM43JA2XVCXplOS9/wH8T0l/mQwPPUbS5K47iIiVwI3AHZJOkzQk+ZwLJF2TbPY88FeShkk6BrhkX4VHxBLgzaSOhyJic/LS08Bbkq6WNFTSQEnvkHTifv5szHbjILBKdBHwo4j4c0S8nnsA3wcuBAR8FDgG+DPQCJwPEBH3AF8jeyrpLbLDQcfsYT+fTz7zBmAz2VNK55Ht1AX4DpAB3gD+m12nefblduCM5F+SujqAc8iG2p/YFRajeviZZnsk35jGzCzd3CIwM0s5B4GZWco5CMzMUs5BYGaWcv1uHsG4ceOirq6u1GWYmfUrzz777JsRMb671/pdENTV1bF48Z7m75iZWXckdXepFMCnhszMUs9BYGaWcg4CM7OUcxCYmaWcg8DMLOWKFgSS/kvSBknL9/C6JH1P0ipJy7q7HaCZmRVfMVsEtwBn7eX1s4GpyWM+8IMi1mJmZntQtHkEEfGYpLq9bHIucGtyV6YnJVVLOiIieuP2f1aB/vjGWzyw9LVSl2FWMqdPr+G4idW9/rmlnFA2gYLb9JG9JvwEurkPrKT5ZFsNTJo0qU+Ks/Lz7795hZ8sWUf21u5m6XPYoVUVFwQ9FhE3ATcB1NfX+wYKKfVmS4bjakfxs8tOLXUpZhWllKOG1pG9UXhObbLOrFubWzOMHj6k1GWYVZxSBsH9wKeT0UMnA1vcP2B709SSYcwwB4FZbyvaqSFJdwCnAeMkNQLXAYMBIuLfgQeBDwOrgFbgM8WqxSpDc4tbBGbFUMxRQ/P28XoAlxZr/1ZZ2to7aMl0MMZBYNbrPLPY+oXNre0AjPapIbNe1y9GDVnlaWxu5abHVtPe0bNBYFu354JgcDHLMkslB4GVxIMvrOfWJ9YybsQhPZ4XMHnsMGYeOaq4hZmlkIPASqKppZ0hAwfwzFdOR54hZlZS7iOwksiOABrsEDArAw4CK4mm1ow7fs3KhIPASqK5JeOhoGZlwkFgJdHky0WYlQ0HgZVEc0vGQ0HNyoRHDRkdncG1P1vOG1t39Nk+N29v93WDzMqEg8B4bfN2bnvqz0yoHsqooX3zLX3WhFG859jxfbIvM9s7B4HR1JIB4PpzZ3L69JoSV2Nmfc19BEZTazYI3Hlrlk4OAqM5aRH4nL1ZOjkILH9qyC0Cs3RyEBjNrRkGDhCHVrnLyCyNHARGc2s7o4f5uj9maeWvgCmyta2dS297jq1tO9+2fu2mFsaPOKREVZlZqTkIUuSl9W/xu5VvctzEaqoL5gtU11ZzxgwPGzVLKwdBiuQ6hb9+3jt8gxczy3MfQYo05+YLeJiomRVwEKSIg8DMuuMgSJHmlgxDBw9k6JCBpS7FzMqIgyBFmlrafTMYM9uNgyBFNrdmqPY9AMysC48aqlCZnZ186uYnWb+lLb9u41s7mDNlTAmrMrNy5CCoUG9sbWPx2mbm1I1h0thh+fUfPe7IElZlZuXIQVChciOE5r/3KE8WM7O9ch9Bhdp1RVH3CZjZ3jkIKtTm1nbAcwbMbN8cBBUq1yLwcFEz2xcHQYVqbs0wQHBolU8NmdneOQgqVHNrhuphQxgwwPcYMLO9cxBUqOaW7M1mzMz2xUFQoZpaMu4fMLMecRBUqNypITOzfXEQVKjm1gxjHARm1gMOggoUEdk+Ap8aMrMecBBUoJZMB5mOTncWm1mPFDUIJJ0l6WVJqyRd083rkyQ9KmmJpGWSPlzMetKiOX95CbcIzGzfihYEkgYCNwBnAzOAeZJmdNlsAXB3RLwTuAC4sVj1pEFnZ7CltZ1Xm1sB3EdgZj1SzKuPzgFWRcRqAEl3AucCLxZsE8ChyfNRwGtFrKfiXfXjZdz7XGN+eewIB4GZ7Vsxg2AC8GrBciNwUpdtvgo8LOlyYDhwRncfJGk+MB9g0qRJvV5opXj5ja38Rc1Izj9xIiOqBnFcbXWpSzKzfqDUncXzgFsiohb4MPD/JO1WU0TcFBH1EVE/fvz4Pi+yv2huaWfmhEP521On8Mn6ib68hJn1SDGDYB0wsWC5NllX6BLgboCIeAKoAsYVsaaK5rkDZnYgihkEzwBTJU2RNIRsZ/D9Xbb5M3A6gKTpZINgYxFrqlht7R20Zjo8UsjM9lvRgiAidgKXAQ8BK8iODmqQdL2kuclmXwQ+K2kpcAdwcUREsWqqZLlbU/pGNGa2v4p6z+KIeBB4sMu6awuevwicUswa0qK5JXtHsjG+NaWZ7SffvL6f2tnRyZpNrfnlF9dvBdwiMLP95yDop77xy5f4j9//abf1hx1aVYJqzKw/cxD0U2ubWplQPZSrz56WXzd62GCmjBtewqrMrD9yEPRTm1szTBozjLnHHVnqUsysnyv1hDI7QL4DmZn1FgdBP9Xc2k61LzNtZr3AQdAPdXYGm1vdIjCz3uEg6Ie2trXTGR4qama9w53F/cyqDdtYvm4LgFsEZtYrHAT9SMuOnZz93cdo78heheOIUZ4zYGYHz0HQj2zalqG9I/jcaUdz1szDmV07qtQlmVkFcBD0I03JheXqJ4/muInVpS3GzCqGO4v7Ed+U3syKwUHQjzQlQeCbz5hZb3IQ9CP5ew64RWBmvchB0I80t2YYOEAcWuWuHTPrPf6LUubWbd7Os2ubAVjWuIXRwwYj+ab0ZtZ7HARl7rqfLefXKzbkl0+YVF26YsysIjkIytwbW3dw0pQxfO28WYAnkZlZ73MQlLmmlgxTa8ZwzGEjSl2KmVUodxaXuebWjIeLmllROQjKWFt7B62ZDg8XNbOichCUsdy8AV9l1MyKyUFQxnIziUf7TmRmVkQOgjL1+pY27lncCPgGNGZWXA6CMvWfv1/NLY+v4ZBBA5g8dnipyzGzCubho2XqzW0ZJlQPZdEX30fV4IGlLsfMKphbBGWqqSXDuBFDHAJmVnQOgjLV3Jqh2n0DZtYHHARlqrk142GjZtYnHARlqrml3aOFzKxPOAjKUGZnJ9t27PT8ATPrEx41VGLL123hgWXr37aurb0D8J3IzKxvOAhK7Ae/fYVfLFvPkEFvb5yNrBrEzCMPLVFVZpYmDoISa9qW4cS60dzzD+8udSlmllLuIyix5taMO4XNrKQcBCXW1OJhomZWWg6CEooITxwzs5IrahBIOkvSy5JWSbpmD9t8UtKLkhok3V7MespNS6aD9o5gzHAPEzWz0tlnZ7Gk4cD2iOhMlgcAVRHRuo/3DQRuAD4INALPSLo/Il4s2GYq8CXglIholnTYgR9K/9Ocv9+AWwRmVjo9GTW0CDgD2JYsDwMeBvY1zGUOsCoiVgNIuhM4F3ixYJvPAjdERDNARGzoeen9S0dn8M2FL/Hmtkx+3ZbtvgOZmZVeT4KgKiJyIUBEbJM0rAfvmwC8WrDcCJzUZZtjAST9ARgIfDUiFnb9IEnzgfkAkyZN6sGuy8/qjdv44WOrGTv87VcUPbZmBDM8X8DMSqgnQdAi6YSIeA5A0l8C23tx/1OB04Ba4DFJsyJic+FGEXETcBNAfX199NK++1TutpPfveCdnDp1XImrMTPbpSdB8AXgHkmvAQIOB87vwfvWARMLlmuTdYUagacioh34k6Q/kg2GZ3rw+f1Kc2s7AKPdMWxmZWafQRARz0iaBvxFsurl5A/3vjwDTJU0hWwAXAB8qss29wHzgB9JGkf2VNHqHtberzS3umPYzMrTPoePSroUGB4RyyNiOTBC0v/Y1/siYidwGfAQsAK4OyIaJF0vaW6y2UPAJkkvAo8CV0XEpgM9mHLW5BFCZlamenJq6LMRcUNuIRnm+Vngxn29MSIeBB7ssu7agucBXJk8KlpzS4ahgwcydIhvPWlm5aUnE8oGSlJuIZkf4K+1+6m5td3DRM2sLPWkRbAQuEvSD5Plvwd+WbySKs83fvkSv/3jBg4fVVXqUszMdtOTILia7Bj+f0iWl5EdOWQ90NEZ/PCxV6gZWcV576wtdTlmZrvZ56mh5NISTwFryM4W/gDZzl/rgS3b24mAv3/fUVxy6pRSl2Nmtps9tggkHUt2aOc84E3gLoCIeH/flFYZcqOF3D9gZuVqb6eGXgJ+B5wTEasAJP1jn1RVQTZ7/oCZlbm9nRr6K2A98KikmyWdTnZmse0Hzx8ws3K3xyCIiPsi4gJgGtnJXl8ADpP0A0ln9lF9/V5+RrEvLWFmZaonncUtEXF7RHyU7PWClpAdSWQ90NSSvRqH+wjMrFz1ZPhoXnLfgPyVQPuz7y1aycLlrxd9Pxve2sGQQQMYOtgzis2sPO1XEFSS+55fR+uODt4xYVRR93Nk9VBm146iYHK2mVlZSW0QNLdk+MjsI/iXj80qdSlmZiVV1JvXl6uOzmDL9nbGeCSPmVk6g2Dr9nY6A0a7A9fMLJ1B0NTq2b5mZjmpDILmZJJXtU8NmZmlMwjy1/9xEJiZpTMINic3kq8e5tm+ZmapDIIdOzsAfNtIMzNSGgSdkf13gCd5mZmlNQiySTDAOWBmltYgyP7ryz6YmaU0CMItAjOzvFQGwa5TQ04CM7OUBkH2XweBmVlqgyCbBM4BM7OUBkG4RWBmlpfKIOjsdIvAzCwnnUHgFoGZWV5Kg8DDR83MclIZBJHvLHYSmJmlMgg6w60BM7OclAZBuH/AzCyR0iBwR7GZWU4qgyAiPHTUzCyRziDALQIzs5xUBkFnZ7iz2Mwskc4gcB+BmVleUYNA0lmSXpa0StI1e9nuryWFpPpi1pPT6T4CM7O8ogWBpIHADcDZwAxgnqQZ3Ww3ErgCeKpYtXQVEQzwuSEzM6C4LYI5wKqIWB0RGeBO4Nxutvtn4F+BtiLW8jY+NWRmtksxg2AC8GrBcmOyLk/SCcDEiPhFEevYTXZCWV/u0cysfJWss1jSAODfgC/2YNv5khZLWrxx48aD3ndn+DpDZmY5xQyCdcDEguXaZF3OSOAdwG8krQFOBu7vrsM4Im6KiPqIqB8/fvxBFxZuEZiZ5RUzCJ4BpkqaImkIcAFwf+7FiNgSEeMioi4i6oAngbkRsbiINQG+1pCZWaGiBUFE7AQuAx4CVgB3R0SDpOslzS3WfnvCncVmZrsMKuaHR8SDwINd1l27h21PK2YthTyPwMxsl1TOLA63CMzM8lIZBB4+ama2S0qDwC0CM7OclAaB+wjMzHJSGQTh4aNmZnmpDILOTp8aMjPLSWcQ+NSQmVleSoPALQIzs5xUBkH2fgSlrsLMrDyk8s+hrzVkZrZLSoPAl6E2M8tJaRB4ZrGZWU4qg8DXGjIz2yWVQeAWgZnZLqkNAvcRmJllpTQIcIvAzCyRyiDwtYbMzHZJZRBkh4+Wugozs/KQ0iBwi8DMLCelQeAJZWZmOakMgvDwUTOzvFQGgU8NmZntks4g6PTwUTOznHQGgSeUmZnlpTIIwhPKzMzyUhkE7iMwM9vFQWBmlnKpDILwzGIzs7xUBoFbBGZmu6Q0CNxZbGaWk9IgcIvAzCwnlUEQvtaQmVleKoPAt6o0M9tlUKkLKAWfGjIrH+3t7TQ2NtLW1lbqUipCVVUVtbW1DB48uMfvSWUQRMCAVLaFzMpPY2MjI0eOpK6uzqdsD1JEsGnTJhobG5kyZUqP35fKP4e+H4FZ+Whra2Ps2LH+f7IXSGLs2LH73bpKZRD4fgRm5cUh0HsO5GeZyiBwH4GZ2S4pDQIcBGZmiaIGgaSzJL0saZWka7p5/UpJL0paJmmRpMnFrCcnez+CvtiTmVn5K1oQSBoI3ACcDcwA5kma0WWzJUB9RMwGfgx8s1j1FAq3CMxsDy6//HImT+6T76Rlo5gtgjnAqohYHREZ4E7g3MINIuLRiGhNFp8EaotYT54nlJlZd9asWcOjjz5KJpPhrbfeKtp+Ojo6ivbZB6KY8wgmAK8WLDcCJ+1l+0uAX3b3gqT5wHyASZMmHXRh7iw2K0//9PMGXnxta69+5owjD+W6j87s0bbXXXcdCxYs4Oabb6ahoYGTTz4ZgNdee43LL7+c1atXs337dm699VZqa2t3Wzdnzhze9a53cfvttzNlyhTWrVvH3LlzefbZZ/nEJz7BmDFjWLp0Keeccw7Tpk3j29/+Ntu3b2fkyJH89Kc/Zfz48d3ua9iwYcyfP5/HH38cgOeee46rrrqKRYsW9crPqCw6iyX9DVAPfKu71yPipoioj4j68ePHH/T+PI/AzLpqaGhg+fLlnH/++UyfPp3ly5cDsHPnTs4++2w+85nPsGTJEp577jmmT5/e7brOzk7Wrl1LXV0dAMuWLWP27NkAvPDCC9TU1PDkk0+yYMEC3v/+9/Pkk0+ydOlSPvjBD3L33XfvcV8zZsxg9erV+ZbElVdeybe+1e2fywNSzBbBOmBiwXJtsu5tJJ0BfAV4X0TsKGI9eZ5HYFaeevrNvRgWLFjA9ddfjySmT59OQ0MDAPfddx/Tp0/nnHPOAWDYsGH8+Mc/3m0dwMqVK5kyZUr+i+ayZcuYNWsWbW1tNDU1ce211+b3d8stt3DXXXexY8cOXn/9db7+9a93u6+cmTNn0tDQwMqVK5k8eTInnHBCrx17MYPgGWCqpClkA+AC4FOFG0h6J/BD4KyI2FDEWt7Gw0fNrNBTTz3FwoULWbJkCZdeeiltbW3MmjULgOeffz5/iiinu3WQ/dafex/A4sWLmT9/Pg0NDZx00kkMGpT9k3vrrbfy9NNP88gjjzBixAje+973MnPmTB544IFuPxfg5JNP5g9/+AM33ngjCxcu7K1DB4p4aigidgKXAQ8BK4C7I6JB0vWS5iabfQsYAdwj6XlJ9xernkLuLDazQl/+8pf5+c9/zpo1a1izZg1Lly7NtwgOP/zw/HOAjRs3drsOoKmpierqagBWrFjBL37xC2bPns0LL7yQP0UE2cB497vfzYgRI7j33nt5/PHHmTVr1h4/F7JBsGDBAs477zwmTJjQq8df1D6CiHgwIo6NiKMj4mvJumsj4v7k+RkRURMRxyePuXv/xF6pyfcjMLO8X//612QyGc4444z8upqaGrZt20ZTUxMXX3wxb7zxBjNnzuT444/niSee6HYdwIc+9CEWLlzIhRdeyD333MPYsWOpqanZLQguvvhibrzxRubMmcOSJUs46qijGD58+B4/F2DatGkccsghXH311b3+M1BE9PqHFlN9fX0sXrz4gN/f2Rkc9eUH+cczjuWKM6b2YmVmdiBWrFjB9OnTS11G2bvssss48cQTueiii/a5bXc/U0nPRkR9d9un5jLUdz/zKjf/bjW52HODwMz6g1deeYWPfOQjnHLKKT0KgQORmiCoHjaYqTUjAJh2+EjOnFlT4orMzPbt6KOP5qWXXirqPlITBGfOPJwzZx5e6jLMzMpOWUwoMzOz0nEQmFnJ9bdBK+XsQH6WDgIzK6mqqio2bdrkMOgFuXsWV1VV7df7UtNHYGblqba2lsbGxrdNnrIDV1VVRW3t/l3I2UFgZiU1ePBgpkyZUuoyUs2nhszMUs5BYGaWcg4CM7OU63fXGpK0EVh7gG8fB7zZi+X0Bz7mdPAxp8PBHPPkiOj2zl79LggOhqTFe7roUqXyMaeDjzkdinXMPjVkZpZyDgIzs5RLWxDcVOoCSsDHnA4+5nQoyjGnqo/AzMx2l7YWgZmZdeEgMDNLudQEgaSzJL0saZWka0pdT2+R9F+SNkhaXrBujKRfSVqZ/Ds6WS9J30t+BssknVC6yg+cpImSHpX0oqQGSVck6yv2uCVVSXpa0tLkmP8pWT9F0lPJsd0laUiy/pBkeVXyel1JD+AASRooaYmkB5Llij5eAElrJL0g6XlJi5N1Rf3dTkUQSBoI3ACcDcwA5kmaUdqqes0twFld1l0DLIqIqcCiZBmyxz81ecwHftBHNfa2ncAXI2IGcDJwafLfs5KPewfwgYg4DjgeOEvSycC/At+JiGOAZuCSZPtLgOZk/XeS7fqjK4AVBcuVfrw574+I4wvmDBT3dzsiKv4BvAt4qGD5S8CXSl1XLx5fHbC8YPll4Ijk+RHAy8nzHwLzutuuPz+AnwEfTMtxA8OA54CTyM4yHZSsz/+eAw8B70qeD0q2U6lr38/jrE3+6H0AeABQJR9vwXGvAcZ1WVfU3+1UtAiACcCrBcuNybpKVRMR65PnrwM1yfOK+zkkpwDeCTxFhR93cprkeWAD8CvgFWBzROxMNik8rvwxJ69vAcb2acEH7/8A/wvoTJbHUtnHmxPAw5KelTQ/WVfU323fj6DCRURIqsgxwpJGAPcCX4iIrZLyr1XicUdEB3C8pGrgp8C00lZUPJLOATZExLOSTitxOX3t1IhYJ+kw4FeSXip8sRi/22lpEawDJhYs1ybrKtUbko4ASP7dkKyvmJ+DpMFkQ+C2iPhJsrrijxsgIjYDj5I9NVItKfeFrvC48secvD4K2NS3lR6UU4C5ktYAd5I9PfRdKvd48yJiXfLvBrKBP4ci/26nJQieAaYmIw6GABcA95e4pmK6H7goeX4R2XPoufWfTkYanAxsKWhu9hvKfvX/T2BFRPxbwUsVe9ySxictASQNJdsnsoJsIHw82azrMed+Fh8HHonkJHJ/EBFfiojaiKgj+//rIxFxIRV6vDmShksamXsOnAksp9i/26XuGOnDDpgPA38ke171K6WupxeP6w5gPdBO9vzgJWTPjS4CVgK/BsYk24rs6KlXgBeA+lLXf4DHfCrZ86jLgOeTx4cr+biB2cCS5JiXA9cm648CngZWAfcAhyTrq5LlVcnrR5X6GA7i2E8DHkjD8SbHtzR5NOT+VhX7d9uXmDAzS7m0nBoyM7M9cBCYmaWcg8DMLOUcBGZmKecgMDNLOQeBWReSOpIrP+YevXa1Wkl1KrhSrFk58CUmzHa3PSKOL3URZn3FLQKzHkquE//N5FrxT0s6JllfJ+mR5HrwiyRNStbXSPppcg+BpZLenXzUQEk3J/cVeDiZKWxWMg4Cs90N7XJq6PyC17ZExCzg+2Svjgnwf4H/jojZwG3A95L13wN+G9l7CJxAdqYoZK8df0NEzAQ2A39d1KMx2wfPLDbrQtK2iBjRzfo1ZG8Oszq56N3rETFW0ptkrwHfnqxfHxHjJG0EaiNiR8Fn1AG/iuwNRpB0NTA4Iv6lDw7NrFtuEZjtn9jD8/2xo+B5B+6rsxJzEJjtn/ML/n0ief442StkAlwI/C55vgj4HORvKjOqr4o02x/+JmK2u6HJncByFkZEbgjpaEnLyH6rn5esuxz4kaSrgI3AZ5L1VwA3SbqE7Df/z5G9UqxZWXEfgVkPJX0E9RHxZqlrMetNPjVkZpZybhGYmaWcWwRmZinnIDAzSzkHgZlZyjkIzMxSzkFgZpZy/x/K2VlgbHg8FwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from sklearn import datasets  \n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 导入数据，分别为输入特征和标签\n",
    "x_data = datasets.load_iris().data  \n",
    "y_data = datasets.load_iris().target \n",
    "\n",
    "# 随机打乱数据\n",
    "np.random.seed(116)  # 使用相同的seed，保证输入特征和标签一一对应\n",
    "np.random.shuffle(x_data)\n",
    "np.random.seed(116)\n",
    "np.random.shuffle(y_data)\n",
    "tf.random.set_seed(116)\n",
    "\n",
    "# 将打乱后的数据集分割为训练集和测试集，训练集为前120行，测试集为后30行\n",
    "x_train = x_data[:-30]\n",
    "y_train = y_data[:-30]\n",
    "x_test = x_data[-30:]\n",
    "y_test = y_data[-30:]\n",
    "\n",
    "# 转换x的数据类型，否则后面矩阵相乘时会因数据类型不一致报错\n",
    "x_train = tf.cast(x_train, tf.float32)\n",
    "x_test = tf.cast(x_test, tf.float32)\n",
    "\n",
    "# from_tensor_slices函数使输入特征和标签值一一对应。（把数据集分批次，每个批次batch组数据）\n",
    "train_db = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(32)\n",
    "test_db = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)\n",
    "\n",
    "# 生成神经网络的参数，4个输入特征故，输入层为4个输入节点；因为3分类，故输出层为3个神经元\n",
    "w1 = tf.Variable(tf.random.truncated_normal([4, 3], stddev=0.1, seed=1))\n",
    "b1 = tf.Variable(tf.random.truncated_normal([3], stddev=0.1, seed=1))\n",
    "\n",
    "lr = 0.1                    # 学习率为0.1\n",
    "train_loss_results = []     # 将每轮的loss记录在此列表中，为后续画loss曲线提供数据\n",
    "test_acc = []               # 将每轮的acc记录在此列表中，为后续画acc曲线提供数据\n",
    "epoch = 500                 # 循环500轮\n",
    "loss_all = 0                # 每轮分4个step，loss_all记录四个step生成的4个loss的和\n",
    "\n",
    "# 训练部分\n",
    "for epoch in range(epoch):  # 数据集级别的循环，每个epoch循环一次数据集\n",
    "    for step, (x_train, y_train) in enumerate(train_db):  # batch级别的循环 ，每个step循环一个batch\n",
    "        with tf.GradientTape() as tape:  # with结构记录梯度信息\n",
    "            y = tf.matmul(x_train, w1) + b1  # 神经网络乘加运算\n",
    "            y = tf.nn.softmax(y)  # 使输出y符合概率分布（此操作后与独热码同量级，可相减求loss）\n",
    "            y_ = tf.one_hot(y_train, depth=3)  # 将标签值转换为独热码格式，方便计算loss和accuracy\n",
    "            # 采用均方误差损失函数mse = mean(sum(y-out)^2)\n",
    "            loss = tf.reduce_mean(tf.square(y_ - y))\n",
    "            loss_all += loss.numpy()  # 将每个step计算出的loss累加，为后续求loss平均值提供数据，这样计算的loss更准确\n",
    "        # 计算loss对各个参数的梯度\n",
    "        grads = tape.gradient(loss, [w1, b1])\n",
    "\n",
    "        # 实现梯度更新 w1 = w1 - lr * w1_grad    b = b - lr * b_grad\n",
    "        w1.assign_sub(lr * grads[0])  # 参数w1自更新\n",
    "        b1.assign_sub(lr * grads[1])  # 参数b自更新\n",
    "\n",
    "    # 每个epoch，打印loss信息\n",
    "    print(\"Epoch {}, loss: {}\".format(epoch, loss_all/4))\n",
    "    train_loss_results.append(loss_all / 4)  # 将4个step的loss求平均记录在此变量中\n",
    "    loss_all = 0  # loss_all归零，为记录下一个epoch的loss做准备\n",
    "\n",
    "    # 测试部分\n",
    "    # total_correct为预测对的样本个数, total_number为测试的总样本数，将这两个变量都初始化为0\n",
    "    total_correct, total_number = 0, 0\n",
    "    for x_test, y_test in test_db:\n",
    "        # 使用更新后的参数进行预测\n",
    "        y = tf.matmul(x_test, w1) + b1\n",
    "        y = tf.nn.softmax(y)\n",
    "        pred = tf.argmax(y, axis=1)  # 返回y中最大值的索引，即预测的分类\n",
    "        # 将pred转换为y_test的数据类型\n",
    "        pred = tf.cast(pred, dtype=y_test.dtype)\n",
    "        # 若分类正确，则correct=1，否则为0，将bool型的结果转换为int型\n",
    "        correct = tf.cast(tf.equal(pred, y_test), dtype=tf.int32)\n",
    "        # 将每个batch的correct数加起来\n",
    "        correct = tf.reduce_sum(correct)\n",
    "        # 将所有batch中的correct数加起来\n",
    "        total_correct += int(correct)\n",
    "        # total_number为测试的总样本数，也就是x_test的行数，shape[0]返回变量的行数\n",
    "        total_number += x_test.shape[0]\n",
    "    # 总的准确率等于total_correct/total_number\n",
    "    acc = total_correct / total_number\n",
    "    test_acc.append(acc)\n",
    "    print(\"Test_acc:\", acc)\n",
    "    print(\"--------------------------\")\n",
    "\n",
    "# 绘制 loss 曲线\n",
    "plt.title('Loss Function Curve')  # 图片标题\n",
    "plt.xlabel('Epoch')  # x轴变量名称\n",
    "plt.ylabel('Loss')  # y轴变量名称\n",
    "# 逐点画出trian_loss_results值并连线，连线图标是Loss\n",
    "plt.plot(train_loss_results, label=\"$Loss$\")\n",
    "plt.legend()  # 画出曲线图标\n",
    "plt.show()  # 画出图像\n",
    "\n",
    "# 绘制 Accuracy 曲线\n",
    "plt.title('Acc Curve')  # 图片标题\n",
    "plt.xlabel('Epoch')  # x轴变量名称\n",
    "plt.ylabel('Acc')  # y轴变量名称\n",
    "plt.plot(test_acc, label=\"$Accuracy$\")  # 逐点画出test_acc值并连线，连线图标是Accuracy\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优缺点\n",
    "- 优点\n",
    "  - 在语音、语义、视觉、各类游戏（如围棋）的任务中表现极好\n",
    "  - 算法可以快速调整，适应新的问题\n",
    "- 缺点\n",
    "  - 需要大量数据进行训练\n",
    "  - 训练要求很高的硬件配置\n",
    "  - 模型处于「黑箱状态」，难以理解内部机制\n",
    "  - 元参数（Metaparameter）与网络拓扑选择困难。"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "d36b7e63cb0f63afce68fe5c49ec0e1190f861a688b361cec73da19541843943"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit ('MachineLearning': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
